Would  Consolidation  of  Army  Software  Engineering  Organizations  Help 


to  Control  Software  Costs  for  Current  and  Future  Systems? 


Gary  M.  Lichvar 


April  16,  2015 


PUBLISHED  BY 

The  Defense  Acquisition  University 
Project  Adviser:  Dr.  Craig  Arndt 
The  Senior  Service  College  Fellowship  Program 
Aberdeen  Proving  Ground,  MD 


11 


Table  of  Contents 


Table  of  Contents . iii 

List  of  Figures . vii 

List  of  Tables . ix 

Abstract . xi 

Chapter  1  -  Introduction . 1 

Overview . 1 

Background . 1 

Problem  Statement . 3 

Purpose  of  This  Study . 3 

Significance  of  This  Research . 3 

Overview  of  the  Research  Methodology . 4 

Research  Questions . 5 

Research  Hypothesis . 5 

Objectives  and  Outcomes . 5 

Limitations  of  the  Study . 5 

Validity  of  the  Research . 6 

Reliability  of  the  Responses . 6 

Chapter  2  -  Literature  Review . 7 

Current  Software  Sustainment  Problem . 7 

Findings  and  Recommendations  from  ASA(ALT) . 8 

Software  Sustainment . 8 

AMC  Software  Support  Transformation  Strategy . 1 1 


iii 


Analysis  of  AMC  Software  Support  Transfonnation  Strategy . 13 

Assessment  of  Software  Sustainment  Processes . 13 

Study  of  Software  Best  Practices . 14 

Evaluation  of  Centralized  Versus  Decentralized  Organizational  Structure . 15 

Literature  Review  Summary . 17 

Chapter  3  -  Research  Methodology . 21 

Research  Hypothesis . 21 

Research  Process . 21 

Data  Collection . 22 

Summary . 23 

Chapter  4  -  Findings . 25 

Population  &  Sample  Size . 25 

Collected  Data . 25 

Demographic  Data . 26 

Summary  of  Demographic  Data . 32 

Questions  Specific  to  Work  and  Organization . 32 

Summary  of  Questions  Specific  to  Work  and  Organization . 39 

Questions  Specific  to  AMC  Current  Software  Engineering  Organizational  Structure . 40 

Summary  of  AMC  Current  Software  Engineering  Organizational  Structure  Questions  ....  44 

Additional  Analysis  of  the  Results . 44 

Chapter  5  -  Conclusions  and  Recommendations . 49 

Conclusions . 49 

Recommendations . 50 


iv 


References 


55 


Glossary  of  Acronyms  and  Terms . 59 

Appendix  A  -  Survey  Instrument . 61 

Appendix  B  -  Responses  to  Question  16 . 69 

Appendix  C  -  Responses  to  Question  21 . 71 


V 


vi 


List  of  Figures 


Figure  1  -  System  Acquisition  Framework . 9 

Figure  2  -  Impact  of  Software  Sustainment  Over  Time . 10 

Figure  3  -  Results  for  Survey  Question  3 . 27 

Figure  4  -  Results  for  Survey  Question  4 . 28 

Figure  5  -  Results  for  Survey  Question  5 . 29 

Figure  6  -  Results  for  Survey  Question  6 . 30 

Figure  7  -  Results  for  Survey  Question  7 . 31 

Figure  8  -  Results  for  Survey  Question  8 . 32 

Figure  9  -  Results  for  Survey  Question  9 . 33 

Figure  10  -  Results  for  Survey  Question  10 . 34 

Figure  1 1  -  Results  for  Survey  Question  11 . 35 

Figure  12  -  Results  for  Survey  Question  12 . 36 

Figure  13  -  Results  for  Survey  Question  13 . 37 

Figure  14  -  Results  for  Survey  Question  14 . 38 

Figure  15  -  Results  for  Survey  Question  15 . 38 

Figure  16  -  Results  for  Survey  Question  17 . 40 

Figure  17  -  Results  for  Survey  Question  18 . 41 

Figure  18  -  Results  for  Survey  Question  19 . 42 

Figure  19  -  Results  for  Survey  Question  20 . 43 

Figure  20  -  Median  Selection  by  Organizational  View . 45 

Figure  2 1  -  Median  Selection  by  IT  Experience . 45 

Figure  22  -  Median  Selection  by  Major  Command  (MACOM) . 46 


vii 


viii 


List  of  Tables 


Table  1  -  Potential  Impact  of  AMC  Reimbursable  Rate  and  Section  219  Funding  in  FY12 


52 


IX 


X 


Abstract 


Software  sustainment  costs  continue  to  rise  as  the  Army  increases  use  of  complex 
software-intensive  systems  to  support  military  operations  and  associated  business  functions. 
Various  studies  have  identified  potential  processes  and  procedures  to  help  control  software  costs; 
however,  no  study  has  been  undertaken  to  determine  whether  organizational  changes  to  the 
Anny  Materiel  Command’s  (AMC)  software  support  centers  can  improve  performance  and 
reduce  costs. 

This  study  attempts  to  detennine  whether  cost  controls  and  improved  software 
management  techniques  can  be  achieved  through  changes  in  AMC’s  software  support 
organizations.  Current  software  sustainment  issues  and  concerns  are  also  examined  to  determine 
whether  organizational  changes  could  address  long-standing  performance  issues  with  software 
development  and  sustainment. 

AMC  software  and  information  technology  (IT)  project  leaders,  supervisors  and 
managers  within  their  software  support  centers  were  surveyed  to  detennine  whether  they 
possessed  the  required  expertise  to  lead  software/IT  projects.  These  software  leaders  were  asked 
whether  their  current  organization  provides  the  resources  necessary  for  their  projects  to  be 
successful  and  whether  the  consolidation  of  software  centers  could  enhance  AMC’s  ability  to 
build  and  maintain  software-intensive  systems. 

Specific  recommendations  to  optimize  software  acquisition,  development,  and 
sustainment  have  been  suggested  and  captured  in  this  study.  The  primary  goal  for  the  study  is  to 
determine  whether  centralization  of  software  sustainment  organizations  can  improve  the 
effectiveness  and  efficiency  of  AMC  software  programs,  thus  minimizing  the  escalation  of 
software  sustainment  costs. 
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Chapter  1  -  Introduction 


Overview 

As  software  sustainment  and  maintenance  costs  escalate  across  the  software  industry,  it 
was  only  a  matter  of  time  before  senior  leaders  began  to  question  the  Army’s  projected  software 
sustainment  costs.  That’s  exactly  what  happened  In  2012,  when  the  Secretary  of  the  Anny  saw 
the  projected  costs  across  the  Program  Objectives  Memorandum  for  the  Post  Production 
Software  Support  (PPSS)  requirements.  While  the  Anny  investigates  multiple  opportunities  to 
address  this  complex  issue,  there  has  yet  to  be  any  real  structural  changes  applied  to  the 
organizations  responsible  for  the  PPSS  mission  within  the  Anny  Materiel  Command.  Could 
structural  changes,  if  implemented  properly,  provide  opportunities  for  the  Anny  to  help  control 
these  escalating  costs?  Software-intensive  systems  will  continue  to  grow  for  the  foreseeable 
future,  so  every  possible  solution  to  reduce  the  long-term  sustainment  costs  of  these  systems 
must  be  examined. 

Background 

In  a  2012  memorandum,  the  Secretary  of  the  Anny  requested  the  Assistant  Secretary  of 
the  Army  for  Acquisition,  Logistics,  and  Technology  (ASA[ALT])  to  recommend  policies, 
procedures,  and  organizational  changes  to  maximize  the  Army’s  ability  to  develop  and  sustain 
software  more  efficiently  (McHugh,  2012).  In  response  to  this  request,  the  ASA(ALT)  prepared 
a  report  (published  in  2013),  and  the  Anny  Materiel  Command  (AMC)  was  asked  to  provide  the 
ASA(ALT)  with  comments  on  the  draft,  which  they  completed  on  October  21,  2013  (Nerger, 
2013).  Although  AMC  (2011)  recommended  organizational  changes  to  their  software  support 
centers  in  an  internal  software  support  transfonnation  strategy,  no  such  recommendations  were 
included  in  their  comments  to  the  ASA(ALT)  (Nerger,  2013). 
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Current  software  support  centers  are  distributed  throughout  the  Anny  Materiel 
Command,  primarily  assigned  to  two  major  commands:  the  Research  Development  and 
Engineering  Command  (RDECOM)  and  the  Communications  and  Electronics  Command 
(CECOM).  The  RDECOM  software  assets  are  decentralized  across  its  subordinate  research  and 
development  centers:  Communications  and  Electronic  Research,  Development  and  Engineering 
Center  (CERDEC);  Aviation  and  Missile  Research,  Development  and  Engineering  Center; 
Armament  Research,  Development  and  Engineering  Center  (ARDEC);  and  Tank  and 
Automotive  Research,  Development  and  Engineering  Center.  CECOM  has  centralized  its 
software  engineering  resources  into  a  single  organization  named  the  CECOM  Software 
Engineering  Center  (SEC). 

While  the  software  tasks  across  the  AMC  software  centers  appear  to  be  similar,  the 
systems  supported  are  quite  different.  Each  organization  specializes  in  specific  Army  domains 
and  programs — aviation  and  missile,  communications  and  electronics,  armaments,  tank  and 
automotive.  In  fiscal  year  (FY)  1998,  consolidation  of  Army  software  resources  did  occur  as  a 
result  of  the  Signal  Organization  Mission  Alignment  (SOMA)/Information  Management 
Functional  Area  Assessment  (IMFAA;  Smith,  1996).  Software  resources  within  the  Infonnation 
Systems  Command  were  realigned  to  the  Communication  and  Electronics  Command.  At  that 
time  the  Software  Engineering  Directorate  from  CERDEC  was  realigned  and  merged  with  these 
new  assets  to  create  the  CECOM  SEC  as  an  organization  directly  reporting  to  the  CECOM 
commanding  general.  This  event  essentially  added  several  combat-support  major  automated 
infonnation  systems  (MAIS)  and  other  infonnation  technology-specific  responsibilities  to  the 
CECOM  SEC  inventory  of  systems.  The  SOMA/IMF AA  study  (Smith,  1996),  combined  with 
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AMC  commander’s  decision  to  consolidate  wholesale  logistics  automation  support,  realigned 
more  than  1,500  military  and  Government  civilian  positions  to  CECOM. 

Problem  Statement 

Would  the  consolidation  of  AMC  software  engineering  organizations  enable  the 
Secretary  of  the  Army  to  maximize  the  Army’s  ability  to  develop  and  sustain  software 
efficiently?  Can  consolidation  enhance  synergy,  eliminate  redundancy,  improve  integration,  and 
improve  prioritization  similar  to  what  was  pursued  with  the  2004  consolidation  of  AMC’s 
research,  development,  and  engineering  centers? 

Purpose  of  This  Study 

The  purpose  of  the  study  is  to  examine  the  impact  of  structural  changes  within  the  AMC 
software  engineering  organizations  to  detennine  whether  structural  changes  can  decrease 
software  sustainment  costs  and  lead  to  better  control  of  software  costs  for  current  and  future 
software-intensive  systems. 

Significance  of  This  Research 

This  study  investigates  the  benefits  and  challenges  of  implementing  organizational 
changes  in  an  effort  to  improve  performance  and  reduce  life-cycle  costs  of  software  sustainment 
within  the  Army.  A  number  of  studies  and  research  papers  have  been  published  that  address 
software  and  infonnation  technology  (IT)  acquisition  shortfalls;  however,  none  of  the  previous 
studies  addressed  the  impact  of  structural/organizational  changes  to  control  software  costs  better 
and  improve  perfonnance.  This  study  adds  specific  knowledge  regarding  the  financial  or 
perfonnance  impact  of  organizational  change  and  optimization  to  the  Army  software  sustainment 
process. 
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Overview  of  the  Research  Methodology 

An  examination  of  published  reports  and  research  papers  was  conducted  to  determine  the 
validity  of  the  research  question  and  hypothesis.  Initial  examination  indicated  interest  in 
identifying  organizational  changes  within  the  Army  that  could  maximize  the  Anny’s  ability  to 
develop  and  sustain  software  efficiently.  The  AMC’s  2011  draft  strategy  did  identify 
organizational  changes;  however,  a  review  of  current  organization  charts  indicates  no  changes 
were  implemented.  Initial  research  also  identified  centralization  as  a  potential  cost  savings.  The 
consolidation  of  the  AMC  research,  development  and  engineering  centers  into  the  RDECOM  in 
2004  was  reviewed  to  detennine  whether  stated  efficiencies  were  achieved  and,  if  so,  whether 
these  could  also  be  possible  with  the  AMC  software  centers/directorates.  General  Kem,  then 
AMC  commander,  stated  the  consolidation  of  AMC  science  and  technology  (S&T)  programs 
“will  enhance  synergy  across  technology  organization,  eliminate  redundancy,  improve  the 
capability  to  do  program  and  system  integration,  and  improve  prioritization  of  programs”  (Kern, 
2003,  p.  2). 

In  addition  to  the  published  information  briefings,  papers,  and  studies,  an  online  survey 
via  SurveyMonkey.com  was  developed.  This  survey  was  used  to  identify  current  practices  at  the 
AMC  software  engineering  centers  and  to  solicit  input  from  existing  IT  project  leaders, 
supervisors,  and  managers.  Approximately  200  AMC  software  leaders  were  targeted  to 
participate  in  the  survey.  A  total  of  40  IT  project  leaders,  supervisors,  and  managers  provided 
responses.  The  survey  was  used  to  collect  current  infonnation  from  AMC  software  organizations 
to  detennine  whether  adequate  practices,  expertise,  and  facilities  exist.  Numerous  Government 
Accountability  Office  (GAO)  reports  detailed  in  the  literature  review  identify  best  practices, 
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which  were  evaluated  against  user  survey  results  to  determine  possible  implementation 
recommendations  for  AMC. 

Research  Questions 

Would  better  software  cost  controls  be  established  and  maintained  if  AMC  consolidates 
software  engineering  centers  into  a  single  major  command?  Will  consolidation  result  in 
improved  software  engineering  sustainment  and  development  processes  that  can  also  address 
shortfalls  identified  in  GAO  and  industry  reports? 

Research  Hypothesis 

Organizational  change  through  consolidation  of  AMC  software  centers/directorates  will 
lead  to  better  control  of  software  costs  for  current  and  future  systems,  as  well  as  more  consistent 
software  practices  across  all  AMC  software  organizations. 

Objectives  and  Outcomes 

There  are  a  number  of  possible  outcomes  based  on  the  infonnation  gathered  and  the 
detailed  literature  review.  Although  the  consolidation  of  software  engineering  assets  across 
Army  Materiel  Command  elements  may  reduce  redundancy,  it  may  not  create  the  cost  controls 
required  to  substantially  lower  the  projected  software  sustainment  costs  identified  by  the 
Secretary  of  the  Army,  or  stabilize  the  software  sustainment  processes  significantly  enough  to 
enhance  cost,  schedule,  and  performance  improvements, 
limitations  of  the  Study 

The  primary  limitation  was  adequate  time  to  collect,  analyze  and  verify  all  the  applicable 
infonnation  from  each  and  every  software  organization  within  AMC.  The  time  limits  of  the 
study  prevented  detailed  exploration;  however,  several  viable  ideas  were  discovered  and 
highlighted.  Limitations  include  assumptions  that  the  work  identified  in  the  software  center’s 
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mission  and  functions  manual  (10-1;  CECOM,  201 1)  as  well  as  in  public  facing  web  pages  is 
accurate  and  currently  being  executed  as  documented.  The  approach  to  consolidation  would  be 
similar  to  the  established  process  that  produced  RDECOM.  The  focus  was  on  the  movement  of 
positions  based  on  the  position  job  series,  not  on  the  work  that  was  actually  being  performed.  For 
the  purposes  of  this  study,  it  is  assumed  these  positions  include  all  those  that  are  supporting  the 
software  development  and  sustainment  missions  currently  being  executed  by  the  individual 
directorates  and  centers. 

Validity  of  the  Research 

Survey  questions  were  provided  to  project  leaders,  supervisors,  and  managers  of  the 
software  engineering  centers/directorates  within  the  Anny  Materiel  Command  to  validate  the 
approach  currently  being  used  to  perfonn  the  software  sustainment  mission  within  AMC.  The 
survey  questions  provided  a  realistic  view  of  the  work  being  perfonned  by  these  organizations, 
and  the  experience  and  expertise  of  AMC’s  software  leaders.  Questions  also  assessed  the  GAO 
and  industry  findings  and  other  literature  compiled  for  this  effort  to  determine  whether  software 
intensive  programs  and  IT  projects  are  experiencing  the  cost,  schedule,  and  perfonnance  issues 
highlighted  by  these  reports. 

Reliability  of  the  Responses 

The  survey  questions  focused  on  objective  criteria  and  were  linked  to  previously  obtained 
infonnation  via  the  organization’s  mission  and  functions  manual,  GAO  reports,  and/or  other 
literature  reviewed/compiled  for  this  effort.  The  nature  of  the  objective  questions  provided  the 
researcher  the  ability  to  assess  similarities  in  a  consistent  fashion. 
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Chapter  2  -  Literature  Review 


A  literature  review  was  first  conducted  to  build  background  knowledge  of  the  issues 
surrounding  the  research  hypothesis  and  question.  Caution  was  taken  to  ensure  the  initial  review 
did  not  bias  the  thinking  or  approach  to  the  study.  After  sufficient  background  information  was 
obtained  and  understood,  the  literature  review  continued  with  a  concentration  on  the  issues  and 
concerns  identified.  During  the  second  phase  of  the  review,  the  infonnation  collected  focused  on 
understanding  the  data  and  assisted  in  organizing  the  material  for  this  study.  This  chapter 
presents  an  assortment  of  infonnation  and  data  to  highlight  the  current  issues  with  the  escalating 
software  sustainment  costs  within  the  Army,  what  organizational  constructs  have  been 
considered  to  improve  perfonnance  and  reduce  cost,  and  the  problems  currently  being 
experienced  in  developing  and  sustaining  large-scale  software-intensive  systems. 

Current  Software  Sustainment  Problem 

Software  sustainment  costs  continue  to  escalate  as  the  use  of  software  grows  in  the  Anny. 
Over  the  last  27  years,  the  number  of  software-intensive  systems  has  increased  by  1,008% 
(ASA[ALT],  2013).  The  number  of  software  lines  of  code  has  increase  by  4,700%  over  the  last 
29  years.  Software  releases  increased  by  634%  over  the  last  12  years  and  software  licenses  have 
increased  by  268%  over  just  the  last  five  years.  PPSS  costs  are  projected  to  increase  by  630% 
over  the  next  14  years.  The  extent  of  the  software  sustainment  cost  increases  failed  to  be 
adequately  projected  just  a  few  years  ago  (ASA[ALT],  2013).  The  costs,  if  uncontrolled,  will  be 
unaffordable.  The  Secretary  of  the  Anny  believes  the  projected  software  sustainment  costs  can 
be  decreased  and  that  carefully  thought  out  changes  in  requirements,  development,  and 
sustainment  processes  can  lead  to  better  control  of  software  costs  for  current  and  future  systems 
(McHugh,  2012).  He  requested  ASA(ALT)  to  recommend  policies,  procedures,  and 
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organizational  changes  that  would  maximize  the  Army’s  ability  to  develop  and  sustain  software 
efficiently. 

Findings  and  Recommendations  from  ASA(ALT) 

In  April  of  2013,  ASA(ALT)  presented  the  Secretary  of  the  Army  the  findings  in  a  report. 
The  ASA(ALT)  report  stated  the  “lack  of  executive  data  and  the  utilization  of  a  valid  cost  model 
leads  to  poor  estimates  of  costs  for  decisions”  (p.  3  ).  The  report  went  on  to  recommend  a 
thorough  review  of  the  portfolio  of  systems  in  order  to  divest  those  systems  no  longer  required  or 
needed.  Although  a  number  of  recommendations  were  suggested  for  policy  and  procedure 
changes  that  could  provide  significant  cost  savings,  no  specific  recommendations  were  made  to 
address  organizational  changes.  In  fact  no  integrated  product  team  (IPT)  was  even  established  to 
consider  organizational  changes. 

Software  Sustainment 

So  what  is  software  sustainment  and  why  is  it  so  costly  to  perfonn?  While  Government 
organizations  refer  to  this  phase  of  the  acquisition  life  cycle  as  sustainment  (Figure  1)  and  view  it 
as  the  last  phase,  the  commercial  software  industry  refers  to  it  as  the  maintenance  phase. 
According  to  Forrester  Research,  the  software  maintenance  phase  can  consume  80-90%  of  the 
total  lifetime  cost  of  software  (Kilner,  2009).  In  2009  Jones  identified  approximately  9  million 
active  software  maintenance  projects  and  only  5  million  software  development  projects.  The  data 
indicates  software  maintenance  is  the  most  expensive  and  time-consuming  aspect  of  any 
software  project,  and  more  companies  today  are  choosing  to  maintain  their  existing  software 
products  rather  than  replacing  them  with  new  software  development  efforts. 
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Figure  1  -  System  Acquisition  Framework 

(Source:  Adapted  from  Defense  Acquisition  Portal 
https  ://dap  .dau.mil/ aphome/ das/P  ages/Default.aspx) 

There  are  a  number  of  work  efforts  involved  in  software  maintenance.  While  Jones 
(2009)  identified  2 1  specific  software  maintenance  functions  that  are  executed  during  this  phase. 
Most  of  these  functions  can  be  placed  into  four  major  categories: 

•  Adaptive  maintenance — keep  the  software  usable  in  a  changing  environment. 
Enhance  the  software  to  make  sure  it  supports  the  changing  needs  of  the  user. 

•  Corrective  maintenance — fix  software  “bugs,”  correcting  identified  problems  in  the 
code. 

•  Perfective  maintenance — improve  perfonnance  or  maintainability  of  the  code. 

•  Preventive  maintenance — correct  latent  faults  in  the  software  before  they  become 
effective  faults. 

If  software  is  not  properly  maintained,  it  can  quickly  become  obsolete.  Figure  2 
illustrates  the  importance  of  the  software  sustainment  process.  If  the  software  is  not  continually 
updated  in  concert  with  the  changes  in  business/mission  need,  the  software  will  degrade  over 
time  and  eventually  become  obsolete,  requiring  a  complete  replacement,  which  is  expensive  and 
time  consuming. 
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(Source  of  Graph:  Software  Engineering  Center) 


Figure  2  -  Impact  of  Software  Sustainment  Over  Time 

(Source:  Adapted  from  CECOM  Software  Engineering  Center  [n.d.]) 


Don  Reifer  of  Reifer  Consultants  (2010)  teamed  with  the  Army  and  Air  Force  to  perform 
a  study  of  software  operations,  maintenance,  and  sustainment — the  mission  of  life-cycle  software 
centers.  The  study  surveyed  200  projects  and  visited  6  Army  and  Air  Force  software  centers. 
Over  70  interviews  were  conducted.  While  the  objective  of  the  study  was  to  find  smarter,  quicker 
and  more  effective  ways  to  maintain  and  sustain  software,  it  provided  a  good  overview  of  the 
work  that  is  performed  at  AMC  software  centers  (Reifer,  2010).  The  study  found  that  70%  of  the 
work  performed  at  these  software  organizations  involved  maintenance,  sustaining  engineering, 
and  independent  verification  and  validation.  The  remaining  30%  was  devoted  to  acquisition 
management  and  software  development.  The  maintenance  team  consists  of  both  Government  and 
in-house  contractor  personnel,  and  the  team  supports  up  to  four  different  software  baselines  in 
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parallel.  The  following  sustainment  functions  and  issues  were  briefed  by  Mr.  Reifer  at  the  March 
2010  Center  for  Systems  and  Software  Engineering  Annual  Research  Review  (Reifer,  2010): 

•  Maintenance  centers  do  more  than  just  software  updates  and  repairs. 

•  Testing  is  the  major  maintenance  activity. 

•  Transition  and  transfer  of  software  systems  is  done  poorly,  which  causes  additional 
work  during  the  sustainment  phase. 

•  Estimates  and  budgets  don’t  cover  all  the  work  that  is  required, 
o  Sustaining  engineering 

o  Product  fielding  and  user  support 
o  Regression  testing 

•  Efficiencies  are  needed  to  cope  with  workload. 

AMC  Software  Support  Transformation  Strategy 

In  201 1,  AMC  surveyed  the  software  organizations  within  their  command  and 
documented  the  findings.  The  survey  was  directed  by  the  executive  deputy  to  the  commanding 
general  to  support  the  Task  Force  Drive  to  Fiscal  Reality  (TF  DFR).  At  the  time,  the  Secretary  of 
the  Anny  directed  AMC  to  conduct  a  review  of  materiel  development  and  sustainment  to  create 
a  more  agile  and  cost-effective  research  development  and  acquisition  system  by  analyzing  work 
flow  and  optimizing  organizations,  processes,  and  procedures  to  support  the  work.  The  report 
analyzed  the  current  operations  of  AMC  software  support  organizations  in  relationship  to 
ASA(ALT)  and  provided  recommendations  for  process  changes,  organizational  realignment,  and 
other  transformative  changes  that  would  better  align  the  ASA(ALT)  and  AMC  customer-supplier 
relationship.  The  objective  was  to  create  an  AMC  organization  that  is  both  effective  and 
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efficient,  and  that  leverages  industry  best  practices  for  the  fast-growing  and  complex  world  of  IT 
and  software  (AMC,  2011). 

Two  separate  data  calls  were  used  to  collect  information  from  all  AMC  software  centers 
regarding  software  functions,  customers,  funding,  IT  workforce,  and  organizational  core 
competencies.  Analysis  of  the  collected  data  identified  a  lack  of  strategic  software  planning 
within  AMC,  poor  governance  of  AMC  software  processes/organizations,  and  deficient  oversight 
of  AMC  software  organizations.  AMC  suggested  the  lack  of  oversight  created  duplication  of 
capabilities  and  increased  costs  resulted  from  these  redundant  capabilities.  A  number  of  strategic 
recommendations  were  made  regarding  software  support,  including  collaboration  with 
ASA(ALT)  to  ensure  synchronization  of  strategic  initiatives  such  as  Common  Operating 
Environment,  Lead  Material  Integrator,  and  DoD  Section  804  IT  Acquisition  Reform  (AMC, 
2011).  Organizational  realignments  were  also  recommended  to  achieve  greater  efficiency  and 
effectiveness  across  both  AMC  and  ASA(ALT). 

The  AMC  strategy  suggested  that  realignments  would  establish  software  support  centers 
of  excellence  by  consolidating  research  and  development  with  the  CECOM  SEC  sustainment 
laboratories  at  Aberdeen  Proving  Ground.  The  strategy  essentially  suggested  the  SOMA/IFMAA 
organization  that  was  created  at  CECOM  in  1998  be  dismantled  and  pieces  be  relocated  to  other 
CECOM  organizations,  with  the  majority  of  the  resources  being  relocated  to  RDECOM 
CERDEC.  While  no  specific  software  support  center  was  identified  to  assume  the  MAIS  that 
came  to  CECOM  in  1998  as  a  result  of  the  SOMA/IFMAA  study,  the  strategy  recommended  that 
this  work,  and  the  resources  to  support  it,  transition  to  the  Program  Executive  Officer  for 
Enterprise  Information  Systems.  Essentially  AMC  was  divesting  itself  from  any  software  support 
for  Army  large-scale  management  infonnation  systems.  While  the  recommended  organizational 
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changes  primarily  affected  the  CECOM  SEC,  AMC’s  largest  software  engineering  center,  to  date 
no  organizational  changes  have  been  implemented.  AMC’s  commitment  to  the  recommended 
changes  in  their  strategy  is  questionable.  When  asked  to  comment  on  ASA(ALT)’s  report  before 
it  was  finalized,  AMC  suggested  no  structural  or  organizational  changes,  nor  offered  insight  into 
any  planned  organizational  changes  within  AMC  (Nerger,  2013). 

Analysis  of  AMC  Software  Support  Transformation  Strategy 

The  Software  Management  Subgroup-TF  DFR  reviewed  the  AMC  (2011)  strategy.  Their 
white  paper  provides  an  assessment  of  the  strategy  and  recommendations  for  further  action  in 
order  to  identify  potential  efficiencies  in  the  areas  of  software  development,  management,  and 
sustainment.  While  the  subgroup  recognized  important  initiatives  and  best  practices  were 
articulated  in  AMC’s  strategy,  they  were  concerned  the  study  did  not  include  input  from  any  of 
the  Anny  Program  Executive  Offices  as  well  as  other  relevant  organizations  across  the  Army. 
The  subgroup  determined  the  strategy  contained  many  good  ideas  and  served  as  a  great  starting 
point  for  discussing  the  complex  area  of  software.  Further,  the  subgroup  recommended  that 
AMC  leadership  establish  a  strategic  working  group  under  the  auspices  of  TF  DFR  to  review  the 
data  collected,  fill  known  data  gaps  as  identified,  and  collectively  develop  recommendations  for 
efficiencies  within  the  TF  DFR  constraints.  The  subgroup  commented  that  the  suggestion  by 
AMC  to  increase  headquarters  staff  to  support  execution  runs  counter  to  the  desire  to  move 
execution  out  of  the  HQ  and  reduce  overhead  (Morrison,  2013). 

Assessment  of  Software  Sustainment  Processes 

A  review  of  GAO  reports  highlights  that  software  and  IT  projects  continue  to  be  viewed 
as  risky,  costly,  and  full  of  unproductive  mistakes  (GAO,  2011).  These  projects  frequently  incur 
cost  overruns  and  schedule  slippages.  GAO  has  recommended  the  services  document  a  standard 
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acquisition  process  that  includes  software  metrics  and  the  necessary  knowledge,  skills,  and 
abilities  required  to  lead  and  manage  software-intensive  programs.  The  GAO  reports  that  DoD’s 
large-scale  software-intensive  system  acquisitions  continue  to  fall  short  of  cost,  schedule,  and 
performance  expectations.  Delays  range  from  2  to  12  years  and  cost  increases  range  from  $530 
million  to  $2.4  billion  (GAO,  2009).  Of  the  nine  success  factors  identified  by  GAO  for  IT 
programs,  two  focused  on  software  organizations/structures:  “#2  Program  staff  had  necessary 
skills  and  #6  Government  and  contractor  staff  was  consistent  and  stable”  (GAO,  201 1,  p.  19). 

While  the  GAO  reports  concentrate  on  the  state  of  government  software  projects  and 
organizations,  the  Standish  Group,  in  their  2013  Chaos  Manifesto,  reported  that  only  39%  of  all 
commercial  software  projects  in  2012  were  delivered  on  time,  on  budget,  and  with  required 
features  and  functions.  The  study  went  on  to  report  that  43%  were  late,  over  budget,  and/or 
delivered  less  than  the  required  features  and  functions,  while  18%  were  cancelled  before 
completion  or  were  delivered  but  never  used  (The  Standish  Group,  2013).  The  Standish  Group 
(2013)  claims  the  2%  improvement  in  the  industry’s  success  rate  noted  in  the  report  can  be 
linked  to  project  management  as  a  profession  and  the  use  of  trained  project  management 
professionals.  These  findings  resemble  those  recommended  in  GAO  reports.  However,  the 
success  comes  with  an  increase  in  project  overhead,  along  with  reduction  in  value  and 
innovation.  The  use  of  project  health  checks,  retrospectives,  dashboards,  and  tracking  systems 
provides  for  an  early  warning  system  so  corrective  actions  can  be  taken  (The  Standish  Group, 
2013). 

Study  of  Software  Best  Practices 

In  2006,  CECOM  SEC  conducted  a  study,  at  the  direction  of  Army  leadership,  to 
detennine  whether  the  Army  software  centers  should  adopt  commercial  best  practices  in  order  to 
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improve  management  of  their  software  engineering  business  processes.  The  study  examined  the 
software  management  practices  of  successful  commercial  companies  and  identified  practices  that 
would  potentially  benefit  the  Army.  The  commercial  companies  selected  for  participation  in  the 
study  were  chosen  because  they  did  not  produce  a  significant  amount  of  Government-driven 
software.  These  companies  provided  a  synopsis  of  software  business  practices  that  are  generally 
independent  of  the  influence  of  Government  acquisition  practices.  While  CECOM  SEC  (2006) 
highlighted  a  number  of  findings,  the  significant  ones  that  are  applicable  to  this  study  include  the 
following: 

•  All  companies  had  clearly  identifiable,  highly  visible  “software  champions.” 

•  All  companies  provide  stable,  well-supported  core  funding  for  their  organic  software¬ 
engineering  centers.  The  lifespan  of  software  ranged  from  10  to  30  years. 

•  Industry  believes  it  is  more  effective  to  sustain  existing  systems  than  build  new  ones. 
Industry  spends  up  to  80%  of  its  annual  software  budget  on  sustainment  activities. 

•  Industry  invests  in  and  maintain  an  in-house,  core-funded  software-engineering 
workforce  to  develop  and  sustain  its  core  software. 

Evaluation  of  Centralized  Versus  Decentralized  Organizational  Structure 

In  centralized  organizations,  the  detailed  operational  decisions  go  to  the  top  for 
resolution.  Whether  the  decision  involves  expense  rates,  hiring  practices,  or  project  negotiation, 
the  final  decision  is  determined  by  the  senior  leader  of  the  organization.  On  many  occasions 
individual  project  leaders  and  division  managers  can  provide  input  to  the  decision  maker,  and 
may  even  be  pennitted  to  provide  recommendations;  however,  the  decision  authority  remains 
with  leadership  of  the  organization.  Centralization  provides  standardization,  consistency,  and 
control  across  the  entire  organization  (Bott,  Coleman,  Eaton,  &  Rowland,  2000).  Alternatively, 
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in  a  decentralized  organization,  as  many  decisions  as  possible  are  settled  at  the  local  level. 
Responsibility  and  authority  is  granted  to  those  closest  to  the  project  and  the  customer  (Bott  et 
ah,  2000). 

There  are  advantages  and  disadvantages  to  both  approaches.  By  transferring  decisions  to 
the  lowest  level  at  which  the  knowledge  and  ability  exists,  it  is  likely  that  better  decisions  will  be 
made  and  performance  will  be  improved  (Bott  et  ah,  2000).  Additionally,  the  motivation  of  the 
supervisors  and  mid-level  managers  are  likely  to  be  improved  by  giving  them  greater 
responsibility  for  the  operation  of  their  organizations.  On  the  other  hand,  this  can  lead  to 
wasteful  duplication.  It  can  also  mean  that  best  practices  will  be  slow  to  spread  throughout  the 
organization.  There  are  many  organizations  in  which  you  find  one  division  using  good  modem 
software  design  methodologies  and  programming  techniques,  while  another  division  is  still  using 
hand-drawn  flowcharts  and  archaic  languages  (Bott  et  ah,  2000). 

As  previously  mentioned,  the  Army  did  consolidate  some  of  its  software  engineering 
organizations  in  1998  as  a  result  of  the  SOMA/IFMAA  study  (Smith,  1996).  In  establishing 
CECOM  SEC,  CECOM  created  the  first  centralized  software  engineering  organization  that 
provided  unified  support  across  the  acquisition  life  cycle  and  established  a  focal  point  for  all 
C4ISR  software  support  (Smith,  1996).  In  2004,  GEN  Kern  transfonned  AMC’s  S&T  programs 
by  creating  RDECOM.  An  advisory  group  made  up  of  Anny  S&T  senior  leaders,  industry, 
academia,  and  other  Services,  recommended  the  establishment  of  the  new  command  in  order  to 
transfonn  AMC’s  S&T  programs  to  align  better  with  the  Anny’s  S&T  vision  (Kem,  2003).  The 
new  AMC  S&T  organization  was  created  to  enhance  synergy  across  technology  organizations, 
eliminate  redundancy,  improve  program  and  system  integration,  and  improve  the  prioritization  of 
programs  (Kem,  2003).  Could  similar  improvements  be  achieved  within  the  Anny  software 
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domain  to  achieve  better  control  of  escalating  software  sustainment  costs  and  substantially 
improve  the  software  business  practices  and  structure  to  meet  the  needs  of  the  Anny  and  their 
program  executive  offices?  Is  it  time  to  break  down  old  barriers  and  transfonn  the  way  the  Anny 
acquires,  develops,  and  sustains  software  intensive  systems? 

While  creating  another  RDECOM-like  organization  may  seem  like  a  logical  step  at  this 
time,  for  the  very  reason  GEN  Kem  implemented  it  in  2004;  we  may  want  to  rethink  that 
strategy  based  on  an  analyses  of  RDECOM’s  first  7  years.  In  201 1  the  Secretary  of  the  Army 
established  an  independent  group  to  study  the  Army’s  acquisition  system.  The  group  investigated 
specific  concerns  that  the  Anny  acquisition  efforts  had  become  less  effective  and  efficient 
(Decker  &  Warner,  2011).  The  panel’s  executive  summary  provided  a  number  of 
recommendations  to  “substantially  alleviate  the  problems  preventing  effective,  efficient  and 
timely  acquisition  of  materiel  and  services  required  by  warfighters”  (Decker  &  Warner,  201 1,  p. 
1).  One  of  those  recommendations  focused  on  the  impact  RDECOM  has  had  on  the  process.  The 
panel  stated  the  expected  benefits  never  materialized  and  recommended  the  individual  research 
development  and  engineering  centers  (RDECs)  be  returned  to  their  respective  Life  Cycle 
Management  Command  (LCMC)  commanders  (Decker  &  Wagner,  2011).  The  panel  found  “no 
evidence  of  major  eliminations  of  redundant  effort,  significant  leveraging  of  defense  and 
commercial  technology  advancements  or  more  products”  (Decker  &  Wagner,  201 1,  p.  1 1). 
Literature  Review  Summary 

The  use  of  software-intensive  systems  across  the  Army  continues  to  grow.  As  the 
software-intensive  system  inventory  grows,  so  do  the  costs  associated  with  sustaining  those 
systems.  Without  appropriate  controls,  oversight,  and  understanding,  the  software  sustainment 
mission  within  the  Army  will  soon  be  unaffordable.  While  several  studies  and  initiatives  have 
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been  undertaken,  no  single  solution  has  been  identified.  It  will  take  a  number  of  actions  across 
the  Anny’s  requirements,  development,  and  sustainment  processes  to  provide  acceptable  relief. 
While  there  has  been  agreement  among  many  of  the  Anny  organizations  on  necessary  process 
changes,  such  as  the  creation  of  a  valid  cost  model  for  estimation  and  the  divestiture  of  systems 
that  provide  duplicate  capabilities,  no  organizational  or  structural  changes  have  been  explored  in 
any  great  detail.  While  AMC  (2011)  examined  organizational  changes  in  response  to 
ASA(ALT)’s  request  for  input,  they  neither  recommended  nor  suggested  any  organizational 
changes. 

Currently  AMC  employs  a  decentralized  organizational  structure  for  their  software 
support  organizations,  with  the  majority  of  the  centers  belonging  to  RDECOM.  While  RDECOM 
has  the  majority  of  individual  software  support  activities,  CECOM  SEC  has  the  largest  software 
center.  The  CECOM  SEC  was  established  in  1996  when  the  SOMA/IFMAA  study  realigned  the 
software  support  activities  from  Information  Systems  Command  to  CECOM,  and  AMC  used  this 
opportunity  to  realign  their  business-information-systems  software  support  to  CECOM. 

Although  centralization  of  AMC  software-support  activities  may  provide  standardization, 
consistency,  and  control,  will  that  be  enough  to  improve  control  of  costs  and  provide  more 
predictable  software  processes.  The  challenge  to  centralization  comes  from  the  findings  in  the 
Decker  and  Wagner  (2011)  study,  which  suggested  the  expected  benefits  from  the  consolidation 
of  the  Anny’s  S&T  organizations  would  never  materialize. 

While  a  number  of  studies  proposed  improvements  to  the  Army’s  software  sustainment 
processes  to  lower  costs,  the  most  comprehensive  recommendations  were  found  in  the 
ASA(ALT)’s  2013  recommendations  to  the  Secretary  of  the  Anny,  highlighted  below: 
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•  Implement  a  process  that  examines  current  system  inventories  and  divest  systems 
where  the  operational  risk  is  justified. 

•  Review  software  sustainment  costs  and  processes  early  in  the  life  cycle — pre- 
Milestone  B. 

•  Improve  the  life-cycle  cost  models  used  to  track  total  cost  of  ownership. 

•  Optimize  software  support  in  the  field. 

•  Establish  processes  to  track  and  manage  software  license  purchases  and  use  across  the 
Anny. 

•  Improve  the  ability  for  the  Anny  to  obtain  data  rights  to  software  developed  by 
vendors. 

•  Improve  guidance  on  the  use  of  COTS  to  include  the  potential  impact  on  long-tenn 
sustainment  costs. 

•  Examine  the  use  of  the  common  operating  environment  and  the  agile  or  similar 
development  processes  to  determine  whether  potential  cost  savings  can  be  achieved. 
Consider  using  Government  resources  for  software  development  and  sustainment,  and 
eliminate  redundancies  between  the  RDEC  and  the  LCMC. 

•  Establish  transparency  in  tracking  PPSS  expenditures. 

•  Improve  the  management  of  software  within  a  delivered  system. 

The  literature  review  provided  good  insight  into  what  the  Army  has  attempted  in  order  to 
improve  control  of  software  costs  and  the  management  of  software  projects.  A  few  studies 
suggested  organizational  changes;  however,  to  date  nothing  has  been  implemented.  This  study 
expects  to  add  additional  insight  regarding  the  potential  contribution  of  organizational  changes, 
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specifically  addressing  the  potential  benefits  derived  from  the  consolidation  of  software 
resources. 
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Chapter  3  -  Research  Methodology 

Research  Hypothesis 

For  this  research  project,  four  potential  research  hypotheses  were  identified: 

•  (Ho)  No  worthwhile  effectiveness  and  efficiencies  can  be  gained  through 
consolidation/centralization. 

•  (Hi)  Major  efficiencies  and  effectiveness  can  be  obtained  through  the  consolidation 
of  software  engineering  support  activities. 

•  (H2)  Some  efficiencies  and  effectiveness  can  be  obtained;  however,  the  limited 
savings  aren’t  worth  the  effort  it  would  take  to  consolidate  the  organizations. 

•  (H3)  Efficiencies  and  effectiveness  of  AMC  software-support  activities  can  be 
achieved;  however,  consolidation/centralization  isn’t  required  in  order  to  obtain 
them. 

Research  Process 

An  examination  of  published  reports  and  research  papers  was  conducted  to  determine  the 
validity  of  the  research  question  and  hypothesis.  Initial  examination  indicated  interest  in 
identifying  organizational  changes  within  the  Army  that  could  maximize  the  Anny’s  ability  to 
develop  and  sustain  software  more  efficiently  (e.g.,  AMC,  2011).  Initial  research  also  identified 
centralization  as  a  potential  cost  savings.  The  consolidation  of  the  AMC  Research,  Development 
and  Engineering  Centers  into  the  Research,  Development  and  Engineering  Command 
(RDECOM)  in  2004  was  reviewed  to  detennine  whether  similar  efficiencies  could  be  achieved 
by  consolidating  the  AMC  software  centers/directorates. 

In  addition  to  the  published  infonnation,  an  online  survey,  via  SurveyMonkey.com,  was 
developed  and  used  to  fill  any  “gaps”  identified  in  the  review  of  published  infonnation.  The 
survey  was  also  used  to  collect  current  information  from  AMC  software  and  IT  project  leaders, 
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supervisors,  and  managers  to  detennine  current  experience,  expertise,  and  perspective.  Questions 
attempted  to  validate  the  work  being  performed  in  the  organization  and  the  resources  available  to 
support  the  software  sustainment  mission.  Numerous  GAO  reports  detailed  in  the  literature 
review  identify  best  practices,  which  were  evaluated  against  user  survey  results  to  detennine 
implementation  recommendations  for  AMC. 

Data  Collection 

Through  research  and  analysis  of  written  and  published  material,  and  the  development 
and  distribution  of  a  SurveyMonkey  questionnaire,  significant  information  was  gathered.  The 
survey  questionnaire  asked  21  questions  (Appendix  A).  Questions  1-8  captured  demographic 
information  about  the  respondents.  Questions  9-16  captured  information  about  the  type  of  work 
the  participant  leads  and  the  support  provided  by  the  individual’s  organization.  Questions  17-21 
addressed  the  participant’s  viewpoints  concerning  the  current  AMC  structure  and  perspective 
regarding  the  benefits  of  centralization.  Two  questions  (#16  and  #21)  were  open-ended.  The 
exact  responses  to  these  questions  are  provided  in  Appendixes  B  and  C. 

The  respondents  provided  essential  data  as  the  key  leaders  required  to  navigate  Anny 
and  AMC  policies  and  practices  on  a  daily  basis  to  ensure  Army  software  projects  are 
successful.  A  pilot  survey  was  initially  developed  and  distributed  to  one  division  within 
CECOM  SEC  to  assess  their  understanding  of  the  questions  and  the  required  responses.  Pilot 
respondents  were  also  asked  to  provide  feedback  regarding  potential  bias  (e.g.,  whether  any  of 
the  questions  were  specific  to  a  particular  organization,  expertise,  or  domain).  The  pilot 
respondents  reported  no  bias  based  on  their  understanding  of  the  questions  and  proposed 
responses.  The  feedback  provided  was  used  to  modify  some  of  the  questions  and  clarify 
potential  responses.  Feedback  from  the  pilot  concentrated  on  the  last  three  questions  of  the 
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survey,  which  focused  on  the  study’s  hypotheses  and  research  question.  Rewording  was 
suggested  to  ensure  respondents  clearly  understood  the  intent  of  these  questions.  The  pilot 
respondents  clearly  understood  these  three  questions  were  the  heart  of  the  survey.  Survey 
questions  included  demographic  data  to  ensure  the  right  audience  was  responding.  One  question 
was  worded  in  a  negative  fashion  specifically  to  test  the  respondent’s  careful  reading  of  each 
question. 

Summary 

The  literature  review  and  survey  information  were  instrumental  in  identifying  the  current 
state  of  AMC  software-support  centers  and  insight  into  their  current  operations.  While  more 
participation  would  have  been  advantageous,  the  responses  coupled  with  the  unsolicited  written 
comments  provided  essential  information  from  those  software  experts  that  understand  the  AMC 
software  domain. 
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Chapter  4  -  Findings 


Population  &  Sample  Size 

Survey  distribution  was  limited  to  software  leaders  in  the  five  AMC  software  engineering 
centers.  Each  center  was  contacted  and  provided  with  an  overview  of  the  study  and  the  required 
participants.  Although  a  count  of  the  exact  number  of  project  leaders,  supervisors,  and  managers 
was  not  possible,  the  anticipated  population  for  this  study  was  approximately  200  individuals 
currently  perfonning  leadership  functions  in  support  of  AMC  software  development  and 
sustainment  projects,  teams,  divisions,  or  organizations.  One  organization  elected  to  limit  the 
distribution  to  only  non-union  members,  which  targeted  supervisors  and  managers.  Another 
elected  to  not  participate  at  all.  No  specific  reason  was  provided  why  the  organization  opted  out. 
The  coordination  effort  required  for  this  particular  survey  was  fairly  extensive  and  took  a 
considerable  amount  of  time.  To  date,  40  responses  were  received  from  three  different  AMC 
software  engineering  organizations.  Considerable  participation  came  from  two  organizations  that 
also  provided  key  infonnation  during  the  literature  review  phase  of  the  study. 

Collected  Data 

As  previously  stated,  40  responses  were  received,  with  one  respondent  electing  to  skip 
multiple  questions.  That  respondent  failed  to  address  any  of  the  questions/statements  in  section 
three  of  the  survey,  dealing  with  viewpoints  regarding  the  current  AMC  structure  and  benefits 
regarding  centralizing  the  software  centers.  Just  two  other  respondents  skipped  one  question 
each,  which  may  have  been  an  oversight.  One  question  that  was  skipped  concerned  demographic 
data  and  another  addressed  AMC’s  current  oversight  of  the  software  centers.  Of  the  possible  760 
responses  (19  questions  per  survey)  for  these  forty  respondents,  754  were  provided — a  99% 
completion  rate.  Two  open-ended,  optional  questions  were  included  in  the  survey  for  the 
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respondents  to  provide  additional  or  clarifying  infonnation.  On  the  first  question,  16  of  the  40 
respondents  provided  responses.  These  are  included  in  Appendix  B.  For  the  second  question,  21 
of  the  40  respondents  provided  responses.  All  21  responses  are  given  in  Appendix  C.  The 
voluntary  comments  were  valuable  in  helping  to  develop  the  study’s  conclusions  and 
recommendations. 

Specific  responses  to  question  1  and  question  2  are  not  recorded  in  the  report.  Question  1 
was  a  grant  of  consent  to  participant  in  the  survey,  and  question  2  was  used  to  identify  the 
participant’s  organization,  which  has  already  been  summarized  above. 

Demographic  Data 

Demographic  data  was  used  to  capture  the  knowledge,  expertise,  and  experience  of  the 
individuals  currently  leading  software  projects,  teams,  divisions,  and  organizations  within  AMC. 
While  software  leaders  can  be  involved  in  a  number  of  different  software  projects,  question  3 
attempted  to  detennine  the  respondent’s  primary  expertise.  A  summary  of  the  responses 
indicates  the  majority  of  experience  is  in  supporting  software  sustainment  activities  or  directly 
providing  software  sustainment  services  (Figure  3).  The  survey  targets  leaders  in  software 
sustainment,  software  development,  and  activities  that  support  software  development  and 
sustainment.  Although  limited,  the  results  indicate  the  projected  audience  for  the  study  was 
obtained. 
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The  majority  of  your  scftware^n  formation  technology  experience  has  been  in 

(select  only  one) 


□  Software  Research 


■  Software  Development 


□  Software  Sustainment 


□  Supporting  Software 
Devetopment'Sustainment 
Activities 

■  Matrix  Support  To  Software/IT 
Programs  of  Record 


Figure  3  -  Results  for  Survey  Question  3 


Question  4  captured  the  number  of  years  the  respondents  have  been  in  their  current 
organizations.  This  is  an  attempt  to  understand  the  respondents’  familiarity  with  how  their 
organizations  operates  and  how  familiar  they  may  be  with  their  current  organizations,  as  well  as 
with  the  AMC  software  engineering  structure.  Figure  4  shows  that  over  50%  of  the  respondents 
have  more  than  10  years  of  experience  in  their  current  organizations,  with  30%  having  more  than 
20  years’  experience  in  their  current  organizations. 
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Please  indicde  number  of  years  in  your  current  organization. 


More  than  20 

Over  15  to  20 

Over  1 0  to  1 5 

5  to  10 

Less  than  5 

0.0%  10.0%  20.0%  30.0%  40.0% 


□  Less  than  5 

■  5  to  10 

□  Over  10  to  15 

□  Over  15  to  20 

■  More  than  20 


Figure  4  -  Results  for  Survey  Question  4 


Question  5  captures  the  years  of  experience  leading  software  and/or  information 
technology  projects.  The  results  indicate  that  more  than  half  of  the  respondents  have  less  than  15 
years’  experience  in  leading  software  projects  (Figure  5).  Many  of  those  have  only  5  to  10  years’ 
experience.  Again,  the  focus  of  question  5  was  the  number  of  years  “leading”  software  and/or 
information  technology  projects.  The  responses  indicate  the  respondents  possess  the  familiarity 
and  experience  necessary  to  deal  with  the  challenges  of  leading  Government  software-intensive 
systems. 
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Please  indicde  your  number  of  years  leading  software/in  formation 
technology  projects  and/or  organizations. 

More  than  20 

Over  15  to  20 

Over  1 0  to  1 5 

5  to  10 

Less  than  5 

0.0%  10.0%  20.0%  30.0%  40.0% 

Figure  5  -  Results  for  Survey  Question  5 

Question  6  provides  an  overview  of  the  respondents’  years  of  Federal  service.  The  data 
reveals  the  majority  of  respondents  have  more  than  20  years  of  Federal  service  (Figure  6).  The 
results  show  the  respondents  are  very  familiar  with  the  Federal  employment  system  and  can  give 
solid  insight  in  identifying  barriers  that  could  be  driving  software  costs  within  the  Army. 


□  Less  than  5 

□  5  to  10 

□  Over  10  to  15 

□  Over  15  to  20 
■  More  than  20 
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Please  indicate  your  number  of  years  of  Federal  Service^ 


More  than  20 

Over  15  to  20 

Over  1 0  to  1 5 

5  to  10 

Less  than  5 
years 

0.0%  20.0%  40.0%  60.0% 


□  Less  than  5  years 

□  5  to  10 

□  Over  10  to  15 

□  Over  15  to  20 
■  More  than  20 


Figure  6  -  Results  for  Survey  Question  6 


Question  7  assesses  the  respondents’  familiarity  with  the  Anny’s  acquisition  process, 
specifically  whether  the  respondent  possessed  the  required  acquisition  career  field  certification 
level.  Level  III  certification  at  this  level  of  leadership  would  be  expected.  The  results  highlighted 
in  Figure  7  confirm  the  respondents  have  a  thorough  understanding  of  the  Anny’s  acquisition 
process  and  are  skilled  acquisition  professionals. 
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Please  indicde  your  highest  Army  Acqisiticn  Certified  ion  Level. 


□  Level  I 

□  Level  II 

□  Level  III 

□  None 


Figure  7  -  Results  for  Survey  Question  7 


As  identified  by  the  results  in  Figure  8,  the  survey  respondents  included  a  good  cross- 
section  of  leaders  from  senior  level  directors  to  nonsupervisory  project  leaders.  However,  the 
project  leader  input  may  be  biased,  because  the  survey  excluded  this  part  of  the  workforce  for 
one  of  the  organizations.  The  respondents  include  a  mix  of  expertise  and  current  experience  at 
different  levels  of  a  software  organization.  The  perspective  presented  is  not  dominated  by  a  view 
of  one  particular  group. 
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Please  identify  your  current  position  within  your  organization. 


Other 
(please 
specify) 

Senior  Level 
Leader  - 
Direct  or/DeL. 

2nd  Level 
Supervisor  - 
Division. 

1st  Level 
Supervisor  - 
Branch/Te_ 

Project 
Leader 
(Non-_ 

0.0%  10.0%  20.0%  30.0%  40.0% 


□  Project  Leader  (Non- 
Supervisor) 

□  1st  Level  Supervisor - 
BranchfTeam  Chief 

□  2nd  Level  Supervisor - 
Division  Chief 

□  Senior  Level  Leader - 
Director/Deputy 

■  Other  (please  specify) 


Figure  8  -  Results  for  Survey  Question  8 


Summary  of  Demographic  Data 

While  the  survey  was  able  to  capture  a  variety  of  leadership  perspectives  ranging  from 
senior  leaders/directors  to  nonsupervisory  project  leaders,  the  majority  of  responses  were 
concentrated  from  two  AMC  software  organizations.  The  years  of  experience  leading  software  or 
information  technology  projects  were  also  widely  distributed  from  5  to  over  20  years.  Based  on 
the  responses  provided,  the  survey  captures  input  from  experienced  acquisition  professionals 
representing  a  long  history  of  work  in  AMC  software  engineering  organizations  performing 
software  sustainment  activities.  The  target  respondent  was  successfully  achieved. 

Questions  Specific  to  Work  and  Organization 

The  overall  goal  of  these  questions  was  to  capture  the  respondents’  assessment  of  their 
current  working  environment.  Does  the  current  AMC  software  engineering  structure  provide  the 
necessary  resources,  tools,  and  systems  required  to  support  the  work  being  performed? 
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Although  subjective,  question  9  attempts  to  determine  whether  the  current  project  leaders 
feel  confident  they  have  the  ability  to  accomplish  their  assigned  missions.  The  results  captured  in 
Figure  9  reveal  the  vast  majority  of  AMC  software  leaders  feel  their  current  organization 
provides  them  the  necessary  structure,  processes,  technology,  and  people  to  be  successful. 


I  have  access  to  the  resources  necessary  (funding,  people,  tools)  for  my 
software  and/or  IT  projects  to  be  successful. 


Stron  gly 
Disagree 

Disagree 

Neutral 

Agree 


Strongly  Agree 


0.0%  20.0%  40.0%  60.0% 


□  Strongly  Agree 

□  Agree 

□  Neutral 

□  Disagree 

■  Strongly  Disagree 


Figure  9  -  Results  for  Survey  Question  9 


Many  may  assume  a  leader’s  authority  is  always  commensurate  with  the  leader’s 
assigned  responsibility;  however,  that  may  not  always  be  the  case.  However,  Figure  10  shows 
the  vast  majority  (79.5%)  of  software  leaders  feel  they  have  been  given  the  appropriate  authority 
to  execute  their  assigned  duties  and  responsibilities. 
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I  have  been  given  the  appropri^e  authority  to  execute  the  duties  and 
responsibilities  assigned  in  order  for  my  software  ancVcr  IT  projects  to  be 

successful. 

Strongly 
Disagree 

Disagree 
Neutral 
Agree 
Strongly  Agree 

0.0%  20.0%  40.0%  60.0% 


□Strongly  Agree 
■Agree 
□Neutral 
□Disagree 
■Strongly  Disagree 


Figure  10  -  Results  for  Survey  Question  10 


While  the  majority  of  leaders  feel  they  have  the  appropriate  authority  to  be  successful, 
the  results  highlighted  in  Figure  1 1  indicate  82%  believe  they  have  the  freedom  and  authority  to 
make  necessary  improvements  to  support  cost  savings  or  process  improvements.  The  previous 
two  questions  indicate  most  leaders  feel  decision  authority  for  software  success  has  been 
delegated  to  the  appropriate  level. 
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I  have  the  freedom  and  authority  to  introdjce  cost  savings  and  process 
improvements  where  necessary  to  support  my  mission 

Stron  gly 
Disagree 

Disagree 
Neutral 
Agree 
Strongly  Agree 

0.0%  20.0%  40.0%  60.0% 


□Strongly  Agree 
■Agree 
□Neutral 
□Disagree 
■Strongly  Disagree 


Figure  11  -  Results  for  Survey  Question  11 


Question  12  is  another  opportunity  to  determine  whether  the  current  organizational 
structure  is  able  to  provide  the  software  projects  a  standard  set  of  tools  and  methods  to  perform 
their  assigned  mission.  The  results  again  indicate  the  current  organizational  structure  provides 
the  tools  and  standards  necessary  to  support  the  successful  implementation  of  their 
responsibilities  (Figure  12). 
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My  organza!) on  provides  a  standard  set  of  tools  and  methorfe  I  employ  to 
perform  my  assigned  software/! T  responsibilities. 


Strongly 
Disagree 

Disagree 
Neutral 
Agree 

Strongly  Agree 

0.0%  20.0%  40.0%  60.0% 


□Strongly  Agree 
■Agree 
□Neutral 
□Disagree 
■Strongly  Disagree 


Figure  12  -  Results  for  Survey  Question  12 


Question  13  is  yet  another  attempt  to  assess  the  responsiveness  of  the  organization  to 
support  the  unique  needs  of  a  particular  project.  The  results  in  Figure  13  again  highlight  the  view 
that  the  current  structure  is  very  responsive  to  the  needs  of  the  individual  projects.  The  strong 
results  indicate  these  leaders  feel  confident  that  their  organization  has  the  resources  necessary  to 
support  emergency  and/or  special  needs  of  their  projects.  The  responses  provide  another 
indication  that  the  organizational  structure  facilitates  the  specific  responsibilities  of  the 
organization.  Resources  and  expertise  are  readily  available  when  needed. 
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If  I  encounter  problems  on  my  software  and/or  I T  projects,  my  organization 
will  provide  resources  to  help  me  analyze  the  cause  and  develop  a  pdh 

forward 


□  Strongly  Agree 
■Agree 

□  Neutral 

□  Disagree 

■  Strongly  Disagree 


Figure  13  -  Results  for  Survey  Question  13 


During  the  literature  review  it  was  suggested  the  individual  software  centers  competed 
with  each  other  for  work.  Questions  14  and  15  were  specifically  included  in  the  survey  to 
determine  the  validity  of  this  assumption.  While  the  responses  to  question  14  clearly  reveal  the 
majority  of  the  work  (89.7%)  being  performed  is  in  direct  support  of  the  LCMC’s  mission 
(Figure  14),  the  results  for  question  15  reflect  only  42.1%  are  discouraged  by  their  leadership  to 
pursue  work  outside  their  immediate  Life  Cycle  Management  Command  (Figure  15).  This  is  the 
only  question  in  the  survey  where  the  neutral  response  (42. 1%)  outscored  every  other  selection. 


37 


The  majority  of  my  current  work  supports  the  domairVcommodties  that  are 
specific  to  my  asskyied  Life  Cyde  Management  Command  (LCMC). 


□  Strongly  Agree 
■Agree 

□  Neutral 

□  Disagree 

■  Strongly  Disagree 


Figure  14  -  Results  for  Survey  Question  14 


I  am  encouraged  by  my  organization  to  seek  work/opportunities  from 
customers  outside  my  assisted  Life  Cyde  Management  Command  (LCMC). 


□  Strongly  Agree 

■  Agree 

□  Neutral 

□  Disagree 

■  Strongly  Disagree 


Figure  15  -  Results  for  Survey  Question  15 
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Question  16  asked  the  respondents  to  provide  additional  comments  or  explanations  for 
their  responses  to  questions  9  through  15.  Sixteen  respondents  elected  to  provide  additional 
comments.  While  the  16  specific  comments  will  not  be  included  here  (see  Appendix  B  for  all 
responses  to  Question  16),  the  following  summary  of  their  comments  highlights  consistent 
topics: 

•  Negative  impact  of  hiring  restrictions  and  resourcing 

•  Customer  demand  for  the  Capability  Maturity  Model  Integrated  (CMMI)  Level  5 
organization  support 

•  CMMI  Level  5  organizations  provide  predictable/well-established  tools  and  processes 

•  Administrative  work  (ITMP,  AKM  Goal  1  Waivers  and  hardware/software 
acquisitions)  consuming  the  majority  of  the  software  expert’s  time 

Summary  of  Questions  Specific  to  Work  and  Organization 

These  questions  centered  on  the  assessment  of  current  work  being  perfonned  and  the 
organizational  support  that  leaders  are  receiving  while  perfonning  their  assigned  duties  and 
responsibilities.  Results  indicate  the  leaders  feel  they  have  the  appropriate  authority  to  execute 
their  responsibilities  and  that  their  organization  is  responsive  to  their  needs.  Standard  tools  and 
processes  are  available  and  used  to  perfonn  the  technical  work.  While  attempts  to  work  beyond 
their  immediate  specialties  are  discouraged,  well-performing  organizations  are  consistently 
sought  out  to  perform  non-LCMC  customer  work.  Specific  comments  captured  in  this  section 
raised  concerns  regarding  recent  fiscal  and  Table  of  Distribution  and  Allowances  (TDA) 
constraints.  These  leaders  are  concerned  that  Government  resources  are  not  available  to  perform 
the  customer  work  being  requested. 
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Questions  Specific  to  AMC  Current  Software  Engineering  Organizational  Structure 

Questions  in  this  area  focused  on  the  study’s  specific  research  question  and  hypothesis. 
Does  the  current  decentralized  structure,  focused  on  specific  commodities,  provide  sufficient 
software  support  for  current  and  future  systems?  Can  the  centralization  of  the  AMC  software 
engineering  assets  deliver  efficiencies  that  result  in  better  control  of  software  sustainment  costs? 

Question  17  asked  whether  the  current  software  engineering  organizational  structure 
within  AMC  is  sufficient  to  support  the  immediate  and  future  software/information  technology 
needs  of  the  Army.  The  responses  are  presented  in  Figure  16.  The  majority  (60.5%)  of  the 
respondents  agreed  the  current  decentralized  structure  was  sufficient  to  support  the  current  and 
future  needs  of  the  Army.  Only  21%  felt  the  organization  was  insufficient. 

The  current  software  engineering  organization^  structure  within  the  Army 

Materiel  Comma 

Strongly 

Disagree 

Disagree 
Neutral 
Agree 

Strongly  Agree 

0.0%  20.0%  40.0%  60.0% 

Figure  16  -  Results  for  Survey  Question  17 

Question  1 8  sought  to  identify  how  active  AMC  is  in  integrating  the  software 
organizations  across  their  command.  Only  16%  of  the  respondents  disagreed  with  this  statement 


□  Strongly  Agree 

■  Agree 

□  Neutral 

□  Disagree 

■  Strongly  Disagree 
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(Figure  17).  The  majority  (62.2%)  of  respondents  indicated  they  agree  that  AMC  is  not  actively 
engaged  with  the  AMC  software  support  centers.  These  responses  are  similar  to  the  findings 
highlighted  by  AMC  (2011).  The  question  is  the  only  negative-directed  question  in  the  survey 
and  was  worded  in  that  way  to  validate  that  the  respondents  were  reading  and  understanding 
each  survey  question.  The  responses  indicate  the  question  was  carefully  read  and  the  response 
was  carefully  selected.  This  provides  confidence  that  the  other  questions  were  also  thoughtfully 
addressed. 


T he  Army  Materiel  Command  (AMC)  lacks  an  intended  approach  to 
software  engineering  and^x  IT  project  r elded  functions  and  processes 
across  their  centers  and  directorates. 


Stron  gly 
Disagree 

Disagree 
Neutral 
Agree 

Strong  Agree 

0.0%  20.0%  40.0%  60.0% 


□  Strong  Agree 

■  Agree 

□  Neutral 

□  Disagree 

■  Strongly  Disagree 


Figure  17  -  Results  for  Survey  Question  18 


Questions  19  and  20  were  included  to  solicit  AMC  software  leaders  regarding  the 
consolidation  or  centralization  of  AMC  software  centers  as  an  effective  way  to  lower  software 
sustainment  costs  and  provide  better  control  over  escalating  software  expenses.  In  Figures  18  and 
19  the  results  indicate  the  respondents’  rejection  of  consolidation  or  centralization  as  a  means  to 
improve  control  of  and  lower  software  sustainment  costs.  Only  15%  of  the  respondents  agreed 
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centralization  can  provide  lower  sustainment  costs,  and  approximately  10%  believe 
consolidation  will  result  in  lower  software  sustainment  costs. 


I  believe  centralizing  software  engineering  functions  within  the  Army  Materiel 
Command  (AMC)  can  lower  the  sustainment  costs  of  Army  software 

intensive  systems. 


Stron  gly 
Disagree 


Disagree 


Neutral 
Agree 
Strongly  Agree 


0.0%  20.0% 


40.0% 


60.0% 


□  Strongly  Agree 

□  Agree 

□  Neutral 

□  Disagree 

■  Strongly  Disagree 


Figure  18  -  Results  for  Survey  Question  19 


42 


T o  provide  better-controlled  software  costs  across  the  Army  Materiel 
Command  (AMC)  I  support  combiningAMC  Software  Engineering  Centers 
and  Directorates  into  a  single  organization. 

Strongly 
Disagree 

Disagree 
Neutral 
Agree 

Strongly  Agree 

0  0%  20.0%  40.0%  60.0% 

Figure  19  -  Results  for  Survey  Question  20 

Question  21  asked  respondents  to  provide  additional  comments  or  explanations.  Twenty- 
one  respondents  elected  to  provide  comments.  While  the  specific  comments  will  not  be  included 
here  (see  Appendix  C  for  all  responses  to  Question  21),  the  following  summary  of  their 
comments  highlights  consistent  topics: 

•  Commodity-focused  software  centers  enhance  workforce  expertise. 

•  Process  improvement,  standardization  (CMMI),  and  state-of-the-art  software 
development  tools/techniques  lower  costs. 

•  Program  managers’  decisions  drive  sustainment  economies. 

•  Centralization  adds  layers,  increases  costs,  and  lowers  productivity. 

•  Collocations  with  LCMCs  allow  cross-fertilization. 

•  LCMCs  collocation  provides  closer  coordination  with  customer  and  enhances 
decisionmaking. 
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•  A  move  toward  Government-owned/Govemment-developed  software  is 
recommended. 

Summary  of  AMC  Current  Software  Engineering  Organizational  Structure  Questions 

This  section  of  the  survey  attempted  to  obtain  feedback  from  those  Federal  employees 
currently  leading  AMC  software  projects,  teams,  and  organizations.  The  primary  objective  was 
to  obtain  their  perspective  and  insight  regarding  the  study’s  research  questions  and  hypothesis. 
The  respondents’  written  comments  were  perhaps  the  most  infonnative.  The  comments 
elucidated  the  rationale  for  their  previous  responses.  While  the  respondents  supported  the  current 
organizational  structure  within  the  AMC  software  centers,  the  majority  agreed  AMC  leadership 
provided  little  insight  about  integrating  the  work  of  the  individual  centers.  It  was  difficult  to 
identify  leaders  who  supported  consolidation  or  centralization  as  a  benefit  in  any  aspect  of 
software  development  or  sustainment.  Their  comments  offered  more  insight  into  their  thought 
process.  Centralization  would  add  layers,  cost,  and  lower  productivity.  Smaller,  more  focused 
organizations  provide  greater  flexibility  to  the  LCMCs,  allowing  them  to  create  and  sustain  the 
required  expertise  to  be  responsive  to  their  customer’s  software  needs.  The  respondents  believe 
standard  processes  and  practices  that  focus  on  software  technologies  and  state-of-the-art 
methodologies  will  create  the  required  economies-of-scale  the  Anny  is  attempting  to  achieve. 
Additional  Analysis  of  the  Results 

Additional  analysis  of  the  data  was  performed  to  determine  whether  particular  views 
were  biased  based  on  the  respondent’s  position  and  experience.  Three  views  were  examined:  (1) 
The  respondent’s  position  in  the  organization  (project  leader/lst  line  supervisor  versus  2nd  line 
supervisor/director),  (2)  The  years  of  IT  experience  (15  years  or  less  versus  more  than  15  years), 
and  (3)  major  command  perspective  (responses  from  those  software  centers  in  RDECOM 
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compared  to  the  software  center  in  CECOM).  The  median  score  was  calculated  for  each  question 
and  a  comparison  of  scores  between  the  identified  groups  was  determined.  The  results  appear  in 
Figures  20,  21,  and  22. 


Organizational  View 

(1)  Strongly  Agree  (2)  Agree  (3)  Netural  (4)  Disagree  (5)  Strongly  Disagree 


Q9  Q10  Qll  Q12  Q13  Q14  Q15  Q17  Q18  Q19  Q20 


■  PL/lstLine  ■  2nd  Line/Dir  ■  Overall 


Figure  20  -  Median  Selection  by  Organizational  View 


Years  of  IT  Experience  View 

(1)  Strongly  Agree  (2)  Agree  (3)  Netural  (4)  Disagree  (5)  Strongly  Disagree 


Q9  Q10  Qll  Q12  Q13  Q14  Q15  Q17  Q18  Q19  Q20 

■  15YrsorLess  ■  More  Than  15  Yrs  ■  Overall 


Figure  21  -  Median  Selection  by  IT  Experience 
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MACOM  View 

(1)  Strongly  Agree  (2)  Agree  (3)  Netural  (4)  Disagree  (5)  Strongly  Disagree 


Q9  Q10  Qll  Q12  Q13  Q14  Q15  Q17  Q18  Q19  Q20 

■  CECOM  ■  RDECOM  ■  Overall 

Figure  22  -  Median  Selection  by  Major  Command  (MACOM) 


The  analysis  revealed  no  significant  difference  in  responses  based  on  where  the 
respondent  worked  in  the  organization  or  the  amount  of  IT  experience  the  respondent  possessed. 
Of  the  1 1  questions  analyzed,  the  median  selections  for  the  majority  of  the  responses  were 
identical.  However,  when  examining  the  responses  based  on  the  major  command,  significant 
differences  appear.  Only  two  of  the  questions  had  similar  responses,  the  remaining  nine  were 
different.  A  significant  statistical  difference  occurred  for  question  9  (“I  have  access  to  the 
resources  necessary  (funding,  people,  tools)  for  my  software  and/or  IT  projects  to  be 
successful”).  While  the  RDECOM  software  centers  agreed  with  the  statement,  the  CECOM 
software  center  disagreed  with  this  statement.  Several  comments  were  received  regarding  the 
ability  to  hire  the  necessary  resources  to  support  the  demand  of  the  work.  This  potential 
resourcing  shortfall  could  account  for  the  wide  gap  between  the  two  organizations.  Additionally, 
funding  shortfalls  in  the  overall  sustainment  program  could  explain  the  gap.  Essentially  the 
overall  assessment  of  the  responses  indicates  a  more  positive  perspective  of  the  current  work 
within  the  RDECOM  software  centers.  Neither  supported  consolidation  as  a  means  to  provide 
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better  control  of  software  sustainment  costs;  not  one  positive  answer  was  provided  by  either 
organization.  In  fact  RDECOM  organizations  were  slightly  more  negative  about  consolidation 
than  the  CECOM  software  center. 
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Chapter  5  -  Conclusions  and  Recommendations 

Conclusions 

The  findings  in  this  study  indicate  consolidation  of  the  Anny  Materiel  Command 
software  support  activities  will  not  provide  significant  improvements  in  the  Anny’s  ability  to 
develop  and  sustain  software  more  efficiently.  AMC’s  existing  decentralized  structure  serves  as 
a  better  foundation  to  implement  required  changes  in  requirements,  development,  and 
sustainment  processes  in  order  to  improve  control  of  software  costs.  The  decentralized  structure 
currently  in  place  actually  centralizes  the  software  development  and  sustainment  activities  by 
commodity.  This  provides  a  number  of  benefits  to  the  individual  software  support  activities, 
particularly  to  their  ability  to  cross-fertilize  the  workforce  in  a  specific  domain  and  expand  upon 
their  expertise.  This  cross-fertilization  process  creates  software  domain  experts  who  possess  both 
a  tactical  and  strategic  perspective.  This  type  of  expertise  was  highlighted  by  GAO  and  industry 
as  a  primary  means  to  improve  the  outcome  of  software-intensive  projects. 

The  decisions  made  by  program  managers  in  the  early  stages  of  a  program’s  life  cycle  are 
the  primary  determinants  of  long-term  software  sustainment  costs.  These  decisions  include  the 
use  of  commercial-off-the-shelf  software  and  equipment,  the  type  of  license  agreement 
implemented,  and  the  type  of  contract  used  to  procure  and  sustain  commercial  products.  While 
consolidating  these  decisions  can  provide  standardization,  consistency,  and  control,  the  decisions 
are  ultimately  the  responsibility  of  the  program  manager,  not  AMC.  As  suggested  in 
ASA(ALT)’s  response  to  the  Secretary  of  the  Army,  AMC  can  provide  the  mechanism  to 
purchase  and  sustain  these  commercial  products  for  the  Anny;  however,  ASA(ALT)  must 
provide  the  guidance  and  policy  to  mandate  its  use. 
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The  survey  results  indicate  the  AMC  software  support  centers  and  the  software-intensive 
projects  they  support  are  led  by  experienced  and  professionally  certified  individuals.  The 
literature  review  that  was  conducted  for  this  study  found  no  specific  examples  of  failures  within 
the  software  centers  that  could  be  attributed  to  the  knowledge,  skills,  and  abilities  of  the 
organization’s  workforce.  As  highlighted  earlier,  one  of  the  reasons  why  software-intensive 
projects  fail  in  both  Government  and  commercial  organizations  can  be  linked  to  the  knowledge 
and  expertise  of  the  project  staff.  The  procedures  and  processes  used  by  the  existing  software 
centers  are  providing  the  expertise  needed  to  ensure  software-intensive  systems  are  staffed  with 
the  knowledge  and  experience  to  make  them  successful.  Current  staffing  strategies  for  software 
activities  support  the  best  practices  highlighted  in  CECOM  SEC’s  (2006)  study:  a  stable  and 
well-trained  workforce  provided  the  company  with  their  competitive  edge. 

While  structural  changes  have  been  identified  in  the  past,  specifically  in  AMC’s  (2011) 
study,  no  structural  changes  have  been  implemented  to  date.  Based  on  the  infonnation  gathered 
through  the  survey  of  AMC  software  and  IT  leadership,  as  well  as  the  assessment  in  Morrison 
(2011)  and  the  feedback  provided  in  the  Decker  and  Wagner  (2011),  one  can  conclude  that 
AMC’s  structural  recommendations  were  questionable  at  best  and  not  in  the  best  interest  of  the 
Army. 

Recommendations 

Sustaining  software-intensive  systems  within  the  Army  will  continue  to  grow  as  more 
and  more  software  systems  are  developed  to  support  the  tactical  and  operational  forces  of  the 
military.  The  ASA(ALT)  recommendations  to  the  Secretary  of  the  Army  were  well  researched, 
comprehensive,  and  realistic.  AMC  should  establish  an  ongoing  relationship  with  ASA(ALT)  to 
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implement  their  2013.  AMC  should  include  their  existing  software  support  centers  in  the 
implementation  planning  and  execution  of  these  recommendations. 

AMC  should  focus  on  developing  policy  that  supports  standardization,  consistency,  and 
control  across  their  software  support  activities.  Policy  should  include  a  standard  set  of  software 
metrics,  software  project  health  checks,  dashboards,  and  other  tracking  systems  to  assure  the 
integrity  and  appropriate  management  of  software-intensive  systems  currently  being  sustained. 
Use  of  commercial  best  practices  and  tools  should  be  encouraged  and  integrated  with  the 
acquisition  life-cycle  model. 

Software  support  activities  that  are  not  in  compliance  with  the  CMMI  (CMMI  Institute, 
2015)  should  develop  an  implementation  strategy  to  become  compliant.  Funding  should  be 
provided  by  AMC  Headquarters  to  support  this  effort,  and  software-support-activity  leadership 
should  be  held  accountable  for  its  timely  implementation.  The  CMMI  is  an  industry-accepted 
model  to  measure  a  software  organization’s  capabilities. 

Additional  studies  should  be  done  to  detennine  the  impact  of  AMC  policies  on  the 
current  software  sustainment  activities  and  the  impact  current  staffing  models  have  on  software 
sustainment  costs.  The  current  AMC  Reimbursable  Support  Rate  Guide,  developed  and 
distributed  by  the  AMC  Deputy  Chief  of  Staff  for  Resource  Management,  G8,  allows  AMC 
organizations  to  establish  their  own  embedded  and  nonembedded  reimbursable  matrix  support 
costs  (Boddorf,  n.d.).  A  review  of  FY12  reimbursable  rates  for  the  AMC  software  support 
activities  reveals  an  annual  reimbursable  rate  that  ranges  from  $5,000  in  one  organization  to 
approximately  $80,000  in  another.  Additionally,  DoD  laboratories  are  permitted  to  collect 
Section  219  Funding.  Section  219  of  the  FY09  National  Defense  Authorization  Act  directs  the 
Secretary  of  Defense  to  develop  a  mechanism  to  allow  DoD  laboratories  to  charge  customers  a 


51 


maximum  of  3%  to  invest  in  infrastructure,  training,  or  research  and  development  (Hunter, 

2008).  Table  1  illustrates  the  effect  these  policies  could  have  on  the  software  support  activities  in 
FY12.  Reimbursable  rates  could  vary  by  as  much  as  80%.  These  policies,  along  with  pay-for- 
perfonnance  systems,  appear  to  be  another  factor  contributing  to  the  rising  cost  of  software 
sustainment.  For  this  study  the  exact  impact  was  not  quantified.  Therefore,  a  follow-on  study  is 
recommended  to  explore  the  impact  of  these  policies  on  the  escalating  software  sustainment 
costs. 

Table  1  -  Potential  Impact  of  AMC  Reimbursable  Rate  and  Section  219  Funding  in  FY12 


AMC 

Software 

Centers 

GS13 

Step  5 

Salary  in 

FY12 

Minimum 

Overhead 

Rate 

Maximum 

Overhead 

Rate 

Minimum 

Salary 

Cost 

Maximum 

Salary 

Cost 

Section 

219 

Funding 

Minimum 

Customer 

Cost 

Maximum 

Customer 

Cost 

SEC  A 

$81,230 

$22,900 

$79,533 

$104,130 

$160,763 

3% 

$107,254 

$165,586 

SEC  B 

$81,230 

$39,520 

$39,520 

$120,750 

$120,750 

3% 

$124,373 

$124,373 

SEC  C 

$81,230 

$29,500 

$42,777 

$110,730 

$124,007 

3% 

$114,052 

$127,727 

SEC  D 

$81,230 

$69,976 

$69,976 

$151,206 

$151,206 

3% 

$155,742 

$155,742 

SECE 

$81,230 

$5,290 

$16,037 

$86,520 

$97,267 

N/A 

$86,520 

$97,267 

It  is  also  evident  the  majority  of  the  software  work  in  AMC  is  performed  by  a  workforce 
that  consists  of  software  engineers  and  computer  scientists.  In  fact,  AMC  (2011)  highlights  this 
as  a  competitive  advantage  of  the  RDECOM  software  support  centers.  While  the  knowledge  of 
relevant  mathematical  and  statistical  sciences  may  be  necessary  for  some  Army  applications,  I 
suspect  the  majority  of  software  systems  in  the  Anny  may  only  require  knowledge  of  IT 
principles,  concepts,  and  methods.  The  knowledge  and  skills  required  to  support  application 
software,  systems  analysis,  data  management,  network  services,  enterprise  architecture,  or 
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systems  administration.  A  separate  follow-on  study  should  determine  whether  AMC  software 
centers  are  staffing  their  responsibilities  with  the  correct  occupation  series  rather  than  staffing 
with  a  more  advanced,  and  more  costly,  occupational  series. 

In  conclusion,  this  study  attempted  to  determine  whether  the  consolidation  of  AMC’s 
software  support  centers  could  provide  significant  cost  savings  for  current  and  future  PPSS. 
While  consolidation  may  initially  eliminate  redundancies,  often  these  efficiencies  are 
outweighed  by  a  reduction  in  effectiveness.  AMC  software  support  centers  are  successful  today 
because  they  are  allowed  to  operate  autonomously  in  conjunction  with  their  commodity-centric 
Life  Cycle  Management  Commands.  These  structures  allow  these  organizations  to  communicate 
directly  with  the  customer  and  to  sustain  a  workforce  that  understands  both  the  tactical  and 
strategic  implications  of  their  decisions.  While  these  autonomous  relationships  may  result  in  a 
variety  of  cost  models  that  could  negatively  impact  the  cost  of  sustaining  Anny  software¬ 
intensive  systems  in  the  long-term,  the  current  decentralized  structure  is  more  effective  for  the 
PPSS  mission.  Until  additional  studies  can  be  undertaken  to  analyze  the  impact  of  the  cost/price 
variance  and  staffing  models,  no  organizational  changes  are  recommended.  As  the  primary 
provider  of  Army  software  expertise,  AMC  should  begin  working  closely  with  their  software 
support  centers  in  order  to  establish  a  comprehensive  strategy  to  address  standardization, 
consistency,  and  control  of  the  ongoing  software  sustainment  activities  within  their  organization. 
AMC  should  also  begin  to  exert  themselves  as  the  “software  champion”  for  the  Army. 
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Glossary  of  Acronyms  and  Terms 


AMC . Army  Material  Command 

ARDEC . Armament  Research  Development  Engineering  Command 

ASA(ALT) . Assistant  Secretary  of  the  Anny  for  Acquisition,  Logistics,  and 

Technology 

CECOM . Communications  Electronics  Command 

CERDEC . Communications  Electronics  Research  Development  Engineering 

Command 

CMMI . Capability  Maturity  Model  Integrated 

FY . fiscal  year 

GAO . Government  Accountability  Office 

Ho . null  hypothesis 

Hi . alternate  hypothesis  1 

H2 . alternate  hypothesis  2 

H3 . alternate  hypothesis  3 

HQ . headquarters 

IFMAA . Infonnation  Management  Functional  Area  Assessment 

IPT . integrated  product  team 

IT . information  technology 

LCMC . Life  Cycle  Management  Command 

MACOM . major  command 

MAIS . major  automated  infonnation  system 

PEO . Program  Executive  Office 
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PPSS . Post  Production  Software  Support 

RDEC . Research  Development  Engineering  Center 

RDECOM . Research  Development  Engineering  Command 

S&T . science  and  technology 

SEC . Software  Engineering  Center 

SOMA . Signal  Organization  and  Mission  Alignment 

TDA . Table  of  Distribution  and  Allowances 

TF  DFR . Task  Force  Drive  to  Fiscal  Reality 
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Appendix  A  -  Survey  Instrument 


Consent 


1.  INFORMED  CONSENT  AGREEMENT 

As  an  adult  18  years  of  age  or  older,  I  agree  to  participate  in  this  research  about  Achieving 
Better-Controlled  Software  Costs  Through  Consolidation  of  AMC  Software  Engineering 
Centers.  This  survey  is  being  conducted  to  support  research  efforts  being  performed  by 
Gary  Lichvar,  a  student  of  the  Senior  Service  College  Fellowship  Program  of  the  Defense 
Acquisition  University. 

1 .  The  purpose  of  the  research  is  to  examine  the  impact  centralization  of  software 
engineering  organizations  might  have  on  controlling  software  costs  within  the  Army 
Materiel  Command.  Should  I  choose  to  participate  in  the  survey,  I  am  aware  that  my 
feedback  will  be  consolidated  with  other  participants  and  the  analysis  will  be  briefed  to 
Army  leadership  allowing  them  to  potentially  take  action  on  my  recommendations. 

2.  If  I  choose  to  participate  in  this  research,  I  will  be  asked  to  continue  the  online 
questionnaire.  The  questionnaire  includes  items  related  to  software  development, 
software  sustainment  and  Information  Technology  (IT)  project  management  and  oversight. 
The  questionnaire  will  take  approximately  15  minutes  to  complete. 

3.  There  is  no  incentive  for  participation. 

4.  All  items  in  the  questionnaire  are  important  for  analysis  and  the  data  will  be  more 
meaningful  if  all  questions  are  answered.  However,  I  do  not  have  to  answer  any  that  I 
prefer  not  to  answer.  I  can  discontinue  my  participation  at  any  time  without  penalty  by 
exiting  out  of  the  survey. 

5.  This  research  will  not  expose  me  to  any  discomfort  or  stress  beyond  that  which  might 
normally  occur  during  a  typical  day.  There  are  no  right  or  wrong  answers;  thus,  I  need  not 
be  stressed  about  finding  a  correct  answer. 

6.  There  are  no  known  risks  associated  with  my  participating  in  this  study. 

7.  Data  collected  will  be  handled  in  a  confidential  manner.  The  data  collected  will  remain 
anonymous. 

8.  The  purpose  of  this  research  has  been  explained  and  my  participation  is  entirely 
voluntary. 
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9. 1  understand  that  the  research  entails  no  known  risks  and  by  completing  this  survey,  I 
am  agreeing  to  participate  in  this  research. 

END  OF  INFORMED  CONSENT 

Survey  Question  #1. 1  have  read  the  Informed  Consent  Agreement  and  will  participate 
voluntarily. 

O  YES 

O  N0 


Specific  To  Your  Work  and  Organization 


2.  Please  identify  your  current  organization. 

o  Armament  Research  Development  and  Engineering  Center  (ARDEC) 
o  Aviation  Missile  Research  Development  and  Engineering  Center  (AMRDEC) 
o  Communications  and  Electronics  Command  (CECOM) 

o  Communications  and  Electronics  Research  Development  and  Engineering  Center  (CERDEC) 
o  Tank  Automotive  Research  Development  and  Engineering  Center  (TARDEC) 

o  Other  (please  specify) 


3.  The  majority  of  your  software/information  technology  experience  has  been  in  (select 
only  one) 

o  Software  Research 
o  Software  Development 
o  Software  Sustainment 

o  Supporting  Software  Development/Sustainment  Activities 
o  Matrix  Support  To  Software/IT  Programs  of  Record 
Other  (please  specify) 
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4.  Please  indicate  number  of  years  in  your  current  organization. 

o  Less  than  5 
Q  5  to  10 
o  Over  10  to  15 
Q  Over  IS  to  20 
o  More  than  20 


5.  Please  indicate  your  number  of  years  leading  software/information  technology  projects 
and/or  organizations. 

o  Less  than  5 

5  to  10 

o  Over  1 0  to  1 5 
o  Over  1 5  to  20 
o  More  than  20 


6.  Please  indicate  your  number  of  years  of  Federal  Service. 

o  Less  than  5  years 

5  to  10 

o  Over  1 0  to  1 5 
o  Over  1 5  to  20 

O  More  than  20 


I  More  than  20 


7.  Please  indicate  your  highest  Army  Acquisition  Certification  Level. 

Level  I 
Level  II 
Q  Level  III 
None 

8.  Please  identify  your  current  position  within  your  organization. 

o  Project  Leader  (Non-Supervisor) 
o  1st  Level  Supervisor  -  Branch/Team  Chief 
o  2nd  Level  Supervisor  -  Division  Chief 
o  Senior  Level  Leader  -  Director/Deputy 
Other  (please  specify) 


Specific  To  Your  Work  and  Organization 
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9. 1  have  access  to  the  resources  necessary  (funding,  people,  tools)  for  my  software  and/or 
IT  projects  to  be  successful. 

(  )  Strongly  Agree 

o  Agree 
o  Neutral 
o  Disagree 
o  Strongly  Disagree 

10.1  have  been  given  the  appropriate  authority  to  execute  the  duties  and  responsibilities 
assigned  in  order  for  my  software  and/or  IT  projects  to  be  successful. 

o  Strongly  Agree 

o  Agree 
o  Neutral 
o  Disagree 
o  Strongly  Disagree 

11. 1  have  the  freedom  and  authority  to  introduce  cost  savings  and  process  improvements 
where  necessary  to  support  my  mission. 

Strongly  Agree 

o  Agree 
o  Neutral 
o  Disagree 

o  Strongly  Disagree 

12.  My  organization  provides  a  standard  set  of  tools  and  methods  I  employ  to  perform  my 
assigned  software/IT  responsibilities. 

Strongly  Agree 
O Aa  ree 

o  Neutral 
o  Disagree 
o  Strongly  Disagree 
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13.  If  I  encounter  problems  on  my  software  and/or  IT  projects,  my  organization  will  provide 
resources  to  help  me  analyze  the  cause  and  develop  a  path  forward. 


o 

o 

o 

o 

o 


Strongly  Agree 
Agree 
Neutral 
Disagree 

Strongly  Disagree 


14.  The  majority  of  my  current  work  supports  the  domain/commodities  that  are  specific  to 
my  assigned  Life  Cycle  Management  Command  (LCMC). 

Strongly  Agree 
Q  Agree 

o  Neutral 
o  Disagree 
o  Strongly  Disagree 


1 5. 1  am  encouraged  by  my  organization  to  seek  work/opportunities  from  customers 
outside  my  assigned  Life  Cycle  Management  Command  (LCMC). 


o  Strongly  Agree 

Agree 

o  Neutral 


o 


Disagree 


o  Strongly  Disagree 


16.  Please  add  any  related  comments/explanations  for  questions  9  - 15. 


AMC  Current  Software  Engineering  Organizational  Structure 
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17.  The  current  software  engineering  organizational  structure  within  the  Army  Materiel 
Command  (AMC)  is  sufficient  to  support  the  immediate  and  future  software/information 
technology  needs  of  the  Army.  (Decentralized  Software  Engineering  Centers  tied  to 
specific  commodities  -  Aviation/Missile,  Armament,  Communication/Electronics, 
Tank/Automotive) 

o  Strongly  Agree 
o  Agree 

O  Neutral 

o  Disagree 
o  Strongly  Disagree 

18.  The  Army  Materiel  Command  (AMC)  lacks  an  integrated  approach  to  software 
engineering  and/or  IT  project  related  functions  and  processes  across  their  centers  and 
directorates. 

Strong  Agree 
([])  Agree 

o  Neutral 
o  Disagree 

O  Strongly  Disagree 

19. 1  believe  centralizing  software  engineering  functions  within  the  Army  Materiel 
Command  (AMC)  can  lower  the  sustainment  costs  of  Army  software  intensive  systems. 

o  Strongly  Agree 
o  Agree 

O  Neutral 
Disagree 

o  Strongly  Disagree 

20.  To  provide  better-controlled  software  costs  across  the  Army  Materiel  Command  (AMC) 
I  support  combining  AMC  Software  Engineering  Centers  and  Directorates  into  a  single 
organization. 

o  Strongly  Agree 
(2)  Agree 
Neutral 

o  Disagree 

O  Strongly  Disagree 
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21.  Please  add  any  additional  comments/explanations  here. 
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Appendix  B  -  Responses  to  Question  16 


#15  -  Seek  new  business  areas  from  outside  established  LCMC  without  lane  violations. 

#9  +  13  -  The  current  fiscal  climate  has  made  it  harder  to  properly  resource  (this  is  above 
AMC  level).  #11-12-Wearea  CMMI  Level  5  organization  and  process  improvement  is  part 
of  our  culture  which  has  reduce  cost  for  SW  Development  as  well  as  supporting  SW 
Acquisitions. 

the  work  I  have  done  is  within  our  lane  of  work  and  within  our  LCMC 

Almost  all  work  is  within  my  assigned  LCMC.  Occasionally,  a  non-customary  customer 
outside  will  approach  my  organization  about  addressing  a  specific  challenge.  This  is  not 
typical  and  is  usually  of  limited  duration. 

Q9  -  Overall,  I  have  access  to  the  resources  that  are  needed.  However,  given  the  current  hiring 
restrictions,  it  is  sometimes  challenging  to  find  the  right  people  for  some  software  projects. 

Q15  - 1  am  encouraged  to  seek  opportunities  from  customers  outside  my  assigned  LCMC 
provided  those  opportunities  do  not  result  in  a  known  "lane  violation."  The  Army  on  the  one 
hand  discourages  "lane  violations"  yet  on  the  other  hand  encourages  competition.  This  is  a 
catch-22.  Organizations  are  not  truly  free  to  seek  opportunities  and  customers  are  not  truly  free 
to  choose  the  organization  that  they  (the  customer)  feel  best  meet  their  needs. 

Having  CECOM  as  the  only  non-RDECOM  center  in  question  #1,  and  specifically  listing  IT 
throughout  the  survey  (as  if  it  is  the  only  application  of  SW)  makes  this  survey  INCREDIBLY 
BIASED  toward  CERDEC.  9.  High  demand  for  ARDEC  SW  services  can  occasionally  out 
pace  our  hiring  authority,  but  our  CMMI  Level  5  organization-wide  processes  ensure  that  SW 
or  Systems  Engineering  expertise  can  be  pulled  from  any  ARDEC  domain  and  reassigned  with 
limited  additional  training  of  the  employee.  10.  The  strong  foundation  of  common  CMMI  level 
5  best  practices  at  ARDEC,  allows  leaders  to  trust  and  empower  our  supervisors  and 
employees  with  little  risk.  1 1 .  The  CMMI  Level  5  process  at  ARDEC  offers  well-established 
methods  that  foster  innovation  and  a  continuous  improvement  mind  set  12.  As  the  only  Federal 
organization  rated  Level  5  in  CMMI,  and  the  only  Federal  Baldridge  Recipient,  the  ARDEC 
SEC  is  a  nationally  recognized  leader  for  its  Organizational  Standard  Processes  (OSP)  15. 
RDECOM  has  well  established  domain  authority  assigned  to  each  of  its  Centers.  As  a  national 
Leader,  ARDEC  frequently  receives  unsolicited  business  opportunities  that  are  not  within  its 
RDECOM  assigned  domain  areas.  In  the  event  that  an  opportunity  falls  within  the  domain  of 
other  Centers,  ARDEC  mandates  that  its  employees  notify  the  responsible  center  so  that  they 
may  capitalize  on  the  opportunity.  In  the  event  that  a  business  opportunity  is  not  assigned  to  a 
particular  Center,  ARDEC  employees  are  directed  to  notify  senior  leadership  to  decide  if  it 
should  be  added  to  ARDECs  portfolio.  However,  our  assigned  domain  areas  and  customers 
always  remain  the  #1  priority. 

15.  We  are  encouraged  to  find  work  outside  our  LCMC  provided  that  it  does  not  violate  any 
pre-positioned  working  lanes  with  other  LCMCs. 
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Within  the  last  year,  we  have  been  highly  discouraged  in  pursuing  new  work/opportunities 
outside  of  the  assigned  LCMC. 

None 

Acquisition  of  IT  hardware  and  software  is  convoluted,  slow,  bureaucratic,  inefficient  and 
takes  too  much  time  away  from  engineers.  The  ITMP,  AKM  Goal  1  Waiver  process,  and 
acquisition  of  HW  &  SW  needs  to  be  streamlined,  expedited  and  much  more  efficient.  If  we 
were  a  private  enterprise  we  would  go  out  of  business.  Also,  management  of  Enterprise 
Software  Licenses  is  broken. 

There  is  more  work  than  can  be  accomplished  by  existing  TDA.  We  are  actually  being  forced 
to  prioritize  and  turn  work  away. 

We  are  not  encouraged  or  instructed  to  seek  work  or  opportunities  from  customers  whose 
mission  is  assigned  to  another  LCMC.  If  a  mission  is  not  assigned  specifically  to  an  LCMC, 
we  can  engage  with  the  customer.  In  addition,  the  customer  is  free  to  choose  who  they  want  to 
work  with  if,  for  example,  they  are  displeased  with  previous  perfonnance. 

My  organization  has  a  well-defined,  mature  process  which  offers  a  set  of  processes  and  tools 
which  are  tailorable  to  each  project.  Common  software  tools  are  available  as  well  as  support  to 
the  projects  to  enable  their  usage.  We  discourage  encroachment  on  other  SEC's  mission  areas. 
My  customers  include  more  than  a  just  a  single  LCMC  - 1  support  both  PEO  Ammo  (JMC 
MSC)  and  PEO  GCSS  (TACOM  LCMC).  I  believe  we  have  customers  outside  the  3  LCMCs 
of  Aviation  &  Missiles,  TACOM,  and  CECOM.  I  am  not  sure  whether  they  approached  us,  or 
we  sought  them  out. 

#11-  Can  only  propose  cost  saving  then  it  is  up  to  decision  makers  to  take  it  to  the  next  level. 

My  organization  gives  me  the  resources  I  need.  There  is  of  course  ways  it  can  work  better,  but 
as  a  CMMI  5  organization,  this  office  has  more  tools  and  standard  processes  than  any  other 
software  organization  I  have  worked  at. 

Quite  often  we  are  approached  by  PM  offices  that  we  have  worked  with  in  the  past  to  support 
new  projects  and  initiatives  they  are  developing. 

There  are  external  influences  that  contribute  to  resource  shortages  ie  hiring  freeze  and  T  DA 
allocations.  The  split  between  CERDEC  and  SEC  has  created  an  additional  layer  of 
bureaucracy  that  stifles  flexibility  and  wastes  resources  on  an  additional  layer  of  management. 
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Appendix  C  -  Responses  to  Question  21 

#18,  #19  &  #20:  A  common  set  of  process  for  the  AMC  SECs  to  follow  would  be  beneficial, 
not  necessarily  a  single  organization,  (e.g.,  all  follow  a  common  set  of  CMMI  Level  5 
processes). 

19.  Each  system  has  different  sustainment  needs.  Further,  each  domain  (Armaments, 
Aviation/Missiles,  Communication/Electronics,  Take/ Automotive)  have  different  needs  and 
technical  challenges.  Combining  could  cause  a  top  heavy  organization  taking  away  the 
specialization  of  each  individual  center.  20  -  Not  sure  if  there  is  any  analysis  that  could  show 
this. 

more  research  would  need  to  be  done  to  determine  how  centralizing  functions  would  affect 
costs 

I  believe  that  commodity-based  Software  Engineering  Centers  allow  better  and  more  efficient 
responses  to  customers'  need  within  the  LCMCs.  Establishing  a  high-level  council  that  meets 
regularly  (quarterly)  could  provide  a  mechanism  for  better  coordination  and  issue  resolution. 
Centralization  would  do  little  to  address  current  issues  and  would  probably  lead  to  a  decrease 
in  efficiency  and  quality  of  work. 

The  different  software  centers,  for  the  most  part,  deal  in  different  domains.  Combining  into  a 
single  organization  will  likely  result  in  sub-optimization.  The  key  to  lowering  development  and 
sustainment  costs  is  in  process  improvement  (using  the  CMMI  model,  for  example)  and 
maximizing  the  use  of  the  latest  software  development  tools  and  techniques.  The  software 
centers  should  strive  for,  at  a  minimum,  a  CMMI  level  3  rating.  Collaboration  among  the 
software  centers  should  be  encouraged  to  the  maximum  extent  possible.  Also,  often,  high 
sustainment  costs  are  a  result  of  decisions  made  by  Program  Managers  without  consulting 
appropriate  software  experts.  Program  Managers  should  be  required  to  consult  software  experts 
for  any  systems  that  contain  software. 

17.  Having  domain  authority  assigned  to  each  RDECOM  center  helps  to  foster  continuous 
improvement  within  the  domain.  It  would  be  advantageous  for  RDECOM  to  establish 
internationally  recognized  STANDARDS  and  PROCESSES  across  all  centers  to  ensure 
consistent,  high  quality  SW  products  and  services  throughout  the  command,  but  to  centralize 
the  SW  function,  or  mandate  poor  practices  (IE  Non-CMMI,  Non-PMI  or  Non-Baldrige)  to 
centers  who  are  already  recognized  leaders  in  these  areas  would  be  a  recipe  for  failure  across 
the  board.  18.  ARDEC  has  frequently  been  called  upon  to  fix  SW/IT  projects  that  have  failed 
due  to  poor  SW  practices,  management  and  execution  by  undisciplined  contractors  and 
Government  agencies  alike.  19.  Centralizing  the  SW  function,  especially  if  centered  around 
organizations  that  are  undisciplined,  too  heavy  in  contractors,  and  with  little  organic  SW 
expertise,  would  be  devastating  to  the  morale,  capability  and  reputation  of  AMC  organizations 
who  are  nationally  recognized  as  true  SW  leaders.  20.  Unless  all  centers  are  required  to  to  be 
assessed  at  CMMI  Level  5,  prior  to  becoming  added  to  the  proposed  centralized  SW 
organization,  control  would  be  lost,  not  gained.  In  many  cases  PMs  and  other  customers  have 
fired  their  contractors  and/or  other  AMC  SW  Centers  and  hired  ARDEC  to  manage  and 
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develop  their  SW  products  in-house  by  ARDEC  personnel,  as  Government  owned/developed. 
This  has  generally  resulted  in  a  significant  cost  savings.  Partnering  with  these  failed 
organization  without  first  improving  their  competency,  should  be  discouraged. 

17.  -  20  -  If  the  Software  Engineering  Centers  were  to  be  combined,  it  would  be  the  most 
efficient  to  allow  ARDEC  to  take  the  lead  since  they  are  the  single  center  that  has  earned  the 
Software  CMMI  level  Five  accreditation.  At  the  attained  level  of  five,  the  rigorous  engineering 
methods  have  proven  themselves  to  attain  high  levels  of  Return  on  Investment  for  projects. 

Centralization  within  the  directorates/organizations  under  AMC  to  this  point  has  proven  to  be 
more  costly  in  that  there  was  no  apparent  analysis  of  the  work  being  performed,  but 
directorates  slashed  to  get  down  to  certain  numbers,  functions  moved  from  one  directorate  to 
another  like  pawns  on  a  chess  board  causing  a  break  in  the  support  links,  an  increased 
workload  on  the  organizations  formed  as  a  result  of  reorganization/centralization  causing 
havoc,  lack  of  support  by  senior  management,  constant  frustration  and  lack  of  morale.  It  will 
take  years  of  work  and  constant  change  to  come  to  a  place  where  what  has  already  occurred 
works  efficiently.  Centralization  has  caused  morale  issues  to  the  very  bottom  of  the  workforce 
with  an  unmanageable  workload  to  get  done.  Things  are  just  falling  off  the  plate.  I  believe  it 
has  led  to  a  number  of  personnel  issues  as  well  due  to  the  stress  the  workforce  feels  and  is 
being  manifested  in  bad  behavior.  The  workload  leaves  very  little  time  to  deal  with  the 
personnel  issues.  So  in  the  long  run,  centralization  cripples  the  very  fiber  of  the  greatest 
resource  needed  for  an  organization  to  be  successful— that  is  the  people.  We  proclaim  people 
are  important,  but  that  is  just  words.  $  are  the  driving  force. 

I  believe  combining  will  increase  costs  because  AMC  will  just  add  layers  of  management  and 
stifle  some  of  what  freedom  exists  in  the  commodity  commands 

Centralization  normally  slows  things  up.  It  is  good  for  the  organization  in  that  they  have 
control,  but  the  user  usually  suffers  because  of  red  tape. 

At  a  minimum,  the  following  needs  to  happen:  1)  a  merging  of  the  SEC  and  SED  (and  even 
related  S&T  organizations)  within  the  C4ISR  community;  2)  keep  each  commodity  are  in  the 
defined  mission  lane. 

I  support  combining  AMC  Software  Engineering  Centers  and  Directorates  into  a  single 
organization  under  the  highest  CMMI  Level  certified  organization.  It  is  only  through  adoption 
of  the  organization  with  the  highest  CMMI  maturity  Level  processes  that  AMC  can  lower  the 
software  development  and  sustainment  costs.  No  organization  should  be  allowed  to  develop  or 
sustain  software  without  at  least  a  CMMI  Level  3,  and  the  minimum  CMMI  level  for  all 
software  centers  must  be  level  5.  If  AMC  is  to  develop  an  integrated  approach  it  only  makes 
logical  sense  to  go  with  the  center  with  CMMI  level  5.  Just  as  we  have  DAU  certifications, 
Information  Assurance  certifications  and  software  developer  certifications  for  individuals 
within  the  workforce,  all  organizations  as  a  whole  should  be  required  to  attain  CMMI 
certification  demonstrating  their  capability  to  develop  and  sustain  software.  CMMI  rated 
organizations  provide  quality,  consistency,  and  continuous  improvement  for  software  intensive 
warfighter  systems. 
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I  believe  that  AMC  is  the  controlling  hand  over  the  SECs  in  matters  which  are  common  to  all, 
and  ensuring  Anny  regs,  public  law,  and  policies  are  implemented.  I  do  not  believe  that 
additional  layers  of  management  at  the  AMC  level  would  decrease  costs.  I  believe  that  costs 
would  increase  if  the  SECs  were  consolidated.  Each  center  is  supporting  a  specific  commodity 
are  with  subject  matter  expertise  built  up  over  the  organization's  history.  My  organization 
provides  very  specific  support  to  armaments,  fire  control  systems,  and  ammunition.  Close 
physical  proximity  to  the  supported  organizations  keeps  TDY  costs  down  by  allowing  project 
IPT  participation  with  no  or  minimal  TDY.  The  migration  of  staff  between  the  commodity 
areas  and  the  SEC,  either  by  details,  rotations,  or  reassignments,  fosters  development  of  subject 
matter  expertise  which  minimizes  costs  and  enhances  efficiency  and  performance.  Any  center 
which  might  have  to  relocate  its  personnel  in  a  consolidation  would  suffer  a  significant  loss  of 
personnel  in  a  move,  with  a  commensurate  loss  in  corporate  knowledge  and  subject  matter 
expertise.  Also,  moving  the  software  engineering  center  away  from  the  customer  base  would 
require  additional  travel  costs  to  both  customer  and  contractor  locations  to  support  projects. 
Putting  all  the  Anny  software  engineering  expertise  in  a  single  geographical  area  would  leave 
it  vulnerable  to  attack  or  natural  disaster. 

In  my  experience,  when  things  are  centralized  it  tend  to  cost  more  in  lost  productivity  and  wait 
times  for  products  or  service  escalate. 

#19  Sustainment  should  be  built  in  during  development  not  during  production  or  post 
production  support. 

Each  software  center  specializes  in  their  field.  Here  at  ARDEC,  our  skills  on  mission  essential 
safety  critical  engineering  is  unmatched  due  to  the  long  length  of  direct  experience. 

Consolidation  of  software  engineering  centers  will  generate  unnecessary  risk  to  the  Software 
Engineering  Centers  and  its  readiness/capability  to  provide  high  quality  products  to  the 
Warfighter. 

Centralizing  functions  usually  sounds  like  a  good  idea,  however  takes  away  direct  customer 
interaction  and  won’t  serve  customers  as  well  or  as  efficiently.  Such  as  centralizing  mail 
services,  what  we  see  is  degraded  service  (from  quick  local  support  to  having  to  go  through 
chain  of  personnel  to  get  issues  resolved.)  and  the  whole  mail  system  seems  to  run  more  slowly 
and  this  can  be  a  single  point  of  failure  issue  when  things  happen. 

17:  the  key  word  to  me  is  "sufficient",  I  think  there  is  always  room  for  improvement.  We 
should  all  be  doing  what  is  best  to  support  the  warfighter’s  needs  with  the  best  resources  and 
abilities  that  AMC  has  to  provide.  18: 1  disagree  because  of  the  word  “lacks”,  I  don't  know  if 
there  is  an  integrated  approach  across  all  centers,  or  if  that  information  has  been  disseminated 
to  all  levels  of  the  work  force.  19:  One  main  factor  to  consider  is  the  cost  of  travel,  to  have  the 
right  personnel  at  the  right  location  at  the  right  time  to  do  the  task.  If  someone  is  local  but  not 
the  subject  matter  expert,  is  that  the  best  thing  to  do?  20: 1  am  not  sure  just  “combining”  will 
work.  I  think  we  need  to  know  what  makes  each  organization  different  and  unique,  and  look 
for  the  best  attributes.  Then  you  have  to  consider  will  all  personnel  be  centralized  and  have 
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additional  travel  to  support  customer  needs.  How  would  a  customer  at  any  Anny  location 
needing  new  software  engineering  support  for  a  new  effort  or  initiative  go  about  getting  that 
support?  I  don’t  know  how  this  would  save  money  and  time  in  getting  a  product  to  the 
warfighter. 

AMC  should  host  on  a  rotating  basis  quarterly  SEC  meetings  to  support  the  warfighter 

layers  of  management  add  costs  to  every  software  release.  Eliminating  redundant  processes 
and  combining  like  efforts,  such  as  IAVA  development  will  lower  lifecycle  costs. 
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